Jul 15 16:56:23.875892 ip-10-0-142-51 systemd[1]: Starting Kubernetes Kubelet...
Jul 15 16:56:24.229174 ip-10-0-142-51 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 16:56:24.229174 ip-10-0-142-51 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 15 16:56:24.229174 ip-10-0-142-51 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 16:56:24.229174 ip-10-0-142-51 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 15 16:56:24.229174 ip-10-0-142-51 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 16:56:24.229174 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.227486    2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 15 16:56:24.306899 ip-10-0-142-51 systemd[1]: Started Kubernetes Kubelet.
Jul 15 16:56:24.465978 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234083    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 16:56:24.465978 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234098    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 16:56:24.465978 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234102    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 16:56:24.465978 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234105    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 16:56:24.465978 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234108    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 16:56:24.465978 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234111    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 16:56:24.465978 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234115    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 16:56:24.465978 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234117    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 16:56:24.465978 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234120    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 16:56:24.465978 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234123    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 16:56:24.465978 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234125    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 16:56:24.465978 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234128    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 16:56:24.465978 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234131    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 16:56:24.465978 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234133    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 16:56:24.465978 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234136    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 16:56:24.465978 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234139    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 16:56:24.465978 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234141    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 16:56:24.465978 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234144    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 16:56:24.465978 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234147    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 16:56:24.465978 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234149    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 16:56:24.467270 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234152    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 16:56:24.467270 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234154    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 16:56:24.467270 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234157    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 16:56:24.467270 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234160    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 16:56:24.467270 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234162    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 16:56:24.467270 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234165    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 16:56:24.467270 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234169    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 16:56:24.467270 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234172    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 16:56:24.467270 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234175    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 16:56:24.467270 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234178    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 16:56:24.467270 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234180    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 16:56:24.467270 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234183    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 16:56:24.467270 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234185    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 16:56:24.467270 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234187    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 16:56:24.467270 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234190    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 16:56:24.467270 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234193    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 16:56:24.467270 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234195    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 16:56:24.467270 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234198    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 16:56:24.467270 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234201    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 16:56:24.467270 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234204    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 16:56:24.467270 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234207    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 16:56:24.468581 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234209    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 16:56:24.468581 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234212    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 16:56:24.468581 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234214    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 16:56:24.468581 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234217    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 16:56:24.468581 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234219    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 16:56:24.468581 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234223    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 16:56:24.468581 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234226    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 16:56:24.468581 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234229    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 16:56:24.468581 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234231    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 16:56:24.468581 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234236    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 16:56:24.468581 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234239    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 16:56:24.468581 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234242    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 16:56:24.468581 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234245    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 16:56:24.468581 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234248    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 16:56:24.468581 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234251    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 16:56:24.468581 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234254    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 16:56:24.468581 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234257    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 16:56:24.468581 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234261    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 16:56:24.468581 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234265    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 16:56:24.469464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234268    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 16:56:24.469464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234271    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 16:56:24.469464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234274    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 16:56:24.469464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234277    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 16:56:24.469464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234279    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 16:56:24.469464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234282    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 16:56:24.469464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234284    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 16:56:24.469464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234287    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 16:56:24.469464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234290    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 16:56:24.469464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234293    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 16:56:24.469464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234295    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 16:56:24.469464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234297    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 16:56:24.469464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234300    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 16:56:24.469464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234302    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 16:56:24.469464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234307    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 16:56:24.469464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234310    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 16:56:24.469464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234313    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 16:56:24.469464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234316    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 16:56:24.469464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234319    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 16:56:24.470347 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234322    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 16:56:24.470347 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234325    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 16:56:24.470347 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234328    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 16:56:24.470347 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234331    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 16:56:24.470347 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234334    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 16:56:24.470347 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234336    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 16:56:24.470347 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234339    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 16:56:24.470347 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234342    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 16:56:24.470347 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234344    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 16:56:24.470347 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234347    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 16:56:24.470347 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234349    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 16:56:24.470347 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234353    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 16:56:24.470347 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234356    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 16:56:24.470347 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234359    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 16:56:24.470347 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234361    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 16:56:24.470347 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234364    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 16:56:24.470347 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234366    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 16:56:24.470347 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234369    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 16:56:24.470347 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234372    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 16:56:24.470347 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.234375    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 16:56:24.471047 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235870    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 16:56:24.471047 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235878    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 16:56:24.471047 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235881    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 16:56:24.471047 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235885    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 16:56:24.471047 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235889    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 16:56:24.471047 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235892    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 16:56:24.471047 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235894    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 16:56:24.471047 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235899    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 16:56:24.471047 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235902    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 16:56:24.471047 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235906    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 16:56:24.471047 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235909    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 16:56:24.471047 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235911    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 16:56:24.471047 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235914    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 16:56:24.471047 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235917    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 16:56:24.471047 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235919    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 16:56:24.471047 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235922    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 16:56:24.471047 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235924    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 16:56:24.471047 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235928    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 16:56:24.471047 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235930    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 16:56:24.471723 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235933    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 16:56:24.471723 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235936    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 16:56:24.471723 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235939    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 16:56:24.471723 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235942    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 16:56:24.471723 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235944    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 16:56:24.471723 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235947    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 16:56:24.471723 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235949    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 16:56:24.471723 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235952    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 16:56:24.471723 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235954    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 16:56:24.471723 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235957    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 16:56:24.471723 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235959    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 16:56:24.471723 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235962    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 16:56:24.471723 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235965    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 16:56:24.471723 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235968    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 16:56:24.471723 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235970    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 16:56:24.471723 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235973    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 16:56:24.471723 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235977    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 16:56:24.471723 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235981    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 16:56:24.471723 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235983    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 16:56:24.471723 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235986    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 16:56:24.472355 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235988    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 16:56:24.472355 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235991    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 16:56:24.472355 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235993    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 16:56:24.472355 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235996    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 16:56:24.472355 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.235998    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 16:56:24.472355 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236001    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 16:56:24.472355 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236004    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 16:56:24.472355 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236007    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 16:56:24.472355 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236009    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 16:56:24.472355 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236012    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 16:56:24.472355 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236014    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 16:56:24.472355 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236018    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 16:56:24.472355 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236021    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 16:56:24.472355 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236025    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 16:56:24.472355 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236028    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 16:56:24.472355 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236031    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 16:56:24.472355 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236034    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 16:56:24.472355 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236036    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 16:56:24.472355 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236039    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 16:56:24.473079 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236042    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 16:56:24.473079 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236045    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 16:56:24.473079 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236047    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 16:56:24.473079 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236050    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 16:56:24.473079 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236054    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 16:56:24.473079 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236057    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 16:56:24.473079 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236060    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 16:56:24.473079 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236063    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 16:56:24.473079 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236065    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 16:56:24.473079 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236068    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 16:56:24.473079 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236071    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 16:56:24.473079 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236073    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 16:56:24.473079 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236076    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 16:56:24.473079 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236078    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 16:56:24.473079 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236081    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 16:56:24.473079 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236083    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 16:56:24.473079 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236086    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 16:56:24.473079 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236088    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 16:56:24.473079 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236091    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 16:56:24.473079 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236093    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 16:56:24.473747 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236096    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 16:56:24.473747 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236098    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 16:56:24.473747 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236101    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 16:56:24.473747 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236104    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 16:56:24.473747 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236107    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 16:56:24.473747 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236111    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 16:56:24.473747 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236113    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 16:56:24.473747 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236116    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 16:56:24.473747 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236119    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 16:56:24.473747 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236121    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 16:56:24.473747 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236124    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 16:56:24.473747 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236126    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 16:56:24.473747 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236129    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 16:56:24.473747 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236132    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 16:56:24.473747 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236134    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 16:56:24.473747 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236137    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 16:56:24.473747 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236139    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 16:56:24.473747 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236142    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 16:56:24.473747 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236144    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 16:56:24.474403 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236147    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 16:56:24.474403 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236149    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 16:56:24.474403 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236548    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 16:56:24.474403 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236553    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 16:56:24.474403 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236577    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 16:56:24.474403 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236582    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 16:56:24.474403 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236586    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 16:56:24.474403 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236590    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 16:56:24.474403 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236593    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 16:56:24.474403 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236596    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 16:56:24.474403 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236602    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 16:56:24.474403 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236605    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 16:56:24.474403 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236608    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 16:56:24.474403 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236611    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 16:56:24.474403 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236614    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 16:56:24.474403 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236617    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 16:56:24.474403 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236620    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 16:56:24.474403 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236622    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 16:56:24.474403 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236626    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 16:56:24.475032 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236628    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 16:56:24.475032 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236632    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 16:56:24.475032 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236635    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 16:56:24.475032 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236638    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 16:56:24.475032 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236640    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 16:56:24.475032 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236643    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 16:56:24.475032 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236646    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 16:56:24.475032 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236648    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 16:56:24.475032 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236651    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 16:56:24.475032 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236654    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 16:56:24.475032 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236657    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 16:56:24.475032 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236660    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 16:56:24.475032 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236662    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 16:56:24.475032 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236665    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 16:56:24.475032 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236667    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 16:56:24.475032 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236670    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 16:56:24.475032 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236673    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 16:56:24.475032 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236675    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 16:56:24.475032 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236677    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 16:56:24.475032 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236680    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 16:56:24.475672 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236683    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 16:56:24.475672 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236685    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 16:56:24.475672 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236687    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 16:56:24.475672 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236692    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 16:56:24.475672 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236695    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 16:56:24.475672 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236697    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 16:56:24.475672 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236700    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 16:56:24.475672 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236702    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 16:56:24.475672 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236705    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 16:56:24.475672 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236707    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 16:56:24.475672 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236710    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 16:56:24.475672 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236712    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 16:56:24.475672 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236716    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 16:56:24.475672 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236720    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 16:56:24.475672 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236724    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 16:56:24.475672 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236726    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 16:56:24.475672 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236729    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 16:56:24.475672 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236732    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 16:56:24.475672 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236735    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 16:56:24.475672 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236738    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 16:56:24.476567 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236741    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 16:56:24.476567 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236743    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 16:56:24.476567 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236747    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 16:56:24.476567 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236751    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 16:56:24.476567 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236753    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 16:56:24.476567 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236756    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 16:56:24.476567 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236758    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 16:56:24.476567 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236760    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 16:56:24.476567 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236763    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 16:56:24.476567 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236765    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 16:56:24.476567 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236768    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 16:56:24.476567 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236771    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 16:56:24.476567 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236773    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 16:56:24.476567 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236775    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 16:56:24.476567 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236778    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 16:56:24.476567 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236782    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 16:56:24.476567 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236784    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 16:56:24.476567 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236787    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 16:56:24.476567 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236791    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 16:56:24.476567 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236793    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 16:56:24.477268 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236796    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 16:56:24.477268 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236798    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 16:56:24.477268 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236801    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 16:56:24.477268 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236803    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 16:56:24.477268 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236807    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 16:56:24.477268 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236810    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 16:56:24.477268 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236813    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 16:56:24.477268 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236815    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 16:56:24.477268 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236818    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 16:56:24.477268 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236821    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 16:56:24.477268 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236823    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 16:56:24.477268 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236826    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 16:56:24.477268 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236828    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 16:56:24.477268 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236831    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 16:56:24.477268 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236833    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 16:56:24.477268 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236836    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 16:56:24.477268 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236838    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 16:56:24.477268 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236841    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 16:56:24.477268 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236843    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236846    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236849    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.236853    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.236916    2575 flags.go:64] FLAG: --address="0.0.0.0"
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.236923    2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.236929    2575 flags.go:64] FLAG: --anonymous-auth="true"
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.236933    2575 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.236938    2575 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.236945    2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.236951    2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.236957    2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.236961    2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.236964    2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.236967    2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.236970    2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.236973    2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.236976    2575 flags.go:64] FLAG: --cgroup-root=""
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.236980    2575 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.236983    2575 flags.go:64] FLAG: --client-ca-file=""
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.236988    2575 flags.go:64] FLAG: --cloud-provider="external"
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.236991    2575 flags.go:64] FLAG: --cluster-dns="[]"
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.236995    2575 flags.go:64] FLAG: --cluster-domain=""
Jul 15 16:56:24.737175 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.236998    2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237001    2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237004    2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237007    2575 flags.go:64] FLAG: --container-log-max-files="5"
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237011    2575 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237014    2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237017    2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237021    2575 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237024    2575 flags.go:64] FLAG: --contention-profiling="false"
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237027    2575 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237030    2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237033    2575 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237036    2575 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237040    2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237043    2575 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237046    2575 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237048    2575 flags.go:64] FLAG: --enable-load-reader="false"
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237051    2575 flags.go:64] FLAG: --enable-server="true"
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237055    2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237059    2575 flags.go:64] FLAG: --event-burst="100"
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237062    2575 flags.go:64] FLAG: --event-qps="50"
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237065    2575 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237068    2575 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237071    2575 flags.go:64] FLAG: --eviction-hard=""
Jul 15 16:56:24.738333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237075    2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237078    2575 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237081    2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237085    2575 flags.go:64] FLAG: --eviction-soft=""
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237088    2575 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237090    2575 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237094    2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237097    2575 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237100    2575 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237103    2575 flags.go:64] FLAG: --fail-swap-on="true"
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237105    2575 flags.go:64] FLAG: --feature-gates=""
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237109    2575 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237112    2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237115    2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237118    2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237121    2575 flags.go:64] FLAG: --healthz-port="10248"
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237124    2575 flags.go:64] FLAG: --help="false"
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237127    2575 flags.go:64] FLAG: --hostname-override="ip-10-0-142-51.ec2.internal"
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237130    2575 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237133    2575 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237136    2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237139    2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237142    2575 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237145    2575 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 15 16:56:24.739089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237148    2575 flags.go:64] FLAG: --image-service-endpoint=""
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237151    2575 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237155    2575 flags.go:64] FLAG: --kube-api-burst="100"
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237158    2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237161    2575 flags.go:64] FLAG: --kube-api-qps="50"
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237164    2575 flags.go:64] FLAG: --kube-reserved=""
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237167    2575 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237169    2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237172    2575 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237175    2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237178    2575 flags.go:64] FLAG: --lock-file=""
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237182    2575 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237185    2575 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237188    2575 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237193    2575 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237196    2575 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237199    2575 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237202    2575 flags.go:64] FLAG: --logging-format="text"
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237205    2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237208    2575 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237211    2575 flags.go:64] FLAG: --manifest-url=""
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237214    2575 flags.go:64] FLAG: --manifest-url-header=""
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237218    2575 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237221    2575 flags.go:64] FLAG: --max-open-files="1000000"
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237225    2575 flags.go:64] FLAG: --max-pods="110"
Jul 15 16:56:24.739860 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237227    2575 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237230    2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237233    2575 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237236    2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237239    2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237242    2575 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237245    2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237252    2575 flags.go:64] FLAG: --node-status-max-images="50"
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237255    2575 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237259    2575 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237262    2575 flags.go:64] FLAG: --pod-cidr=""
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237265    2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237271    2575 flags.go:64] FLAG: --pod-manifest-path=""
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237274    2575 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237277    2575 flags.go:64] FLAG: --pods-per-core="0"
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237280    2575 flags.go:64] FLAG: --port="10250"
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237283    2575 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237286    2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ce0f9454cf8246d1"
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237291    2575 flags.go:64] FLAG: --qos-reserved=""
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237294    2575 flags.go:64] FLAG: --read-only-port="10255"
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237297    2575 flags.go:64] FLAG: --register-node="true"
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237299    2575 flags.go:64] FLAG: --register-with-taints=""
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237303    2575 flags.go:64] FLAG: --registry-burst="10"
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237307    2575 flags.go:64] FLAG: --registry-qps="5"
Jul 15 16:56:24.740604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237309    2575 flags.go:64] FLAG: --reserved-cpus=""
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237312    2575 flags.go:64] FLAG: --reserved-memory=""
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237316    2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237319    2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237321    2575 flags.go:64] FLAG: --rotate-certificates="false"
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237324    2575 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237327    2575 flags.go:64] FLAG: --runonce="false"
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237330    2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237333    2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237336    2575 flags.go:64] FLAG: --seccomp-default="false"
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237339    2575 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237341    2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237344    2575 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237347    2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237350    2575 flags.go:64] FLAG: --storage-driver-password="root"
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237353    2575 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237356    2575 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237359    2575 flags.go:64] FLAG: --storage-driver-user="root"
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237364    2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237367    2575 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237370    2575 flags.go:64] FLAG: --system-cgroups=""
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237373    2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237378    2575 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237381    2575 flags.go:64] FLAG: --tls-cert-file=""
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237384    2575 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 15 16:56:24.741331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237396    2575 flags.go:64] FLAG: --tls-min-version=""
Jul 15 16:56:24.742065 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237400    2575 flags.go:64] FLAG: --tls-private-key-file=""
Jul 15 16:56:24.742065 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237403    2575 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 15 16:56:24.742065 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237406    2575 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 15 16:56:24.742065 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237409    2575 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 15 16:56:24.742065 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237412    2575 flags.go:64] FLAG: --v="2"
Jul 15 16:56:24.742065 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237420    2575 flags.go:64] FLAG: --version="false"
Jul 15 16:56:24.742065 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237425    2575 flags.go:64] FLAG: --vmodule=""
Jul 15 16:56:24.742065 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237429    2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 15 16:56:24.742065 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.237432    2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 15 16:56:24.742065 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237573    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 16:56:24.742065 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237577    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 16:56:24.742065 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237580    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 16:56:24.742065 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237584    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 16:56:24.742065 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237587    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 16:56:24.742065 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237589    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 16:56:24.742065 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237592    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 16:56:24.742065 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237595    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 16:56:24.742065 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237597    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 16:56:24.742065 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237600    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 16:56:24.742065 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237603    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 16:56:24.742065 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237605    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 16:56:24.742065 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237608    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 16:56:24.742764 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237610    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 16:56:24.742764 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237613    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 16:56:24.742764 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237618    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 16:56:24.742764 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237621    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 16:56:24.742764 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237623    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 16:56:24.742764 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237626    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 16:56:24.742764 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237628    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 16:56:24.742764 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237631    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 16:56:24.742764 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237633    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 16:56:24.742764 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237636    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 16:56:24.742764 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237640    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 16:56:24.742764 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237642    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 16:56:24.742764 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237646    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 16:56:24.742764 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237650    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 16:56:24.742764 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237653    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 16:56:24.742764 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237656    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 16:56:24.742764 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237659    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 16:56:24.742764 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237662    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 16:56:24.742764 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237665    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 16:56:24.743331 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237668    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 16:56:24.743331 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237671    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 16:56:24.743331 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237674    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 16:56:24.743331 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237676    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 16:56:24.743331 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237680    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 16:56:24.743331 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237683    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 16:56:24.743331 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237685    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 16:56:24.743331 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237688    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 16:56:24.743331 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237690    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 16:56:24.743331 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237693    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 16:56:24.743331 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237695    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 16:56:24.743331 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237697    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 16:56:24.743331 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237700    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 16:56:24.743331 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237702    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 16:56:24.743331 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237705    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 16:56:24.743331 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237710    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 16:56:24.743331 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237713    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 16:56:24.743331 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237715    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 16:56:24.743331 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237718    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 16:56:24.743947 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237720    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 16:56:24.743947 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237723    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 16:56:24.743947 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237726    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 16:56:24.743947 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237728    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 16:56:24.743947 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237732    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 16:56:24.743947 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237734    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 16:56:24.743947 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237737    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 16:56:24.743947 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237740    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 16:56:24.743947 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237742    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 16:56:24.743947 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237745    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 16:56:24.743947 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237747    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 16:56:24.743947 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237750    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 16:56:24.743947 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237753    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 16:56:24.743947 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237756    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 16:56:24.743947 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237758    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 16:56:24.743947 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237762    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 16:56:24.743947 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237766    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 16:56:24.743947 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237769    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 16:56:24.743947 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237772    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 16:56:24.743947 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237775    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 16:56:24.744598 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237777    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 16:56:24.744598 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237780    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 16:56:24.744598 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237782    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 16:56:24.744598 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237784    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 16:56:24.744598 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237787    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 16:56:24.744598 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237790    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 16:56:24.744598 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237792    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 16:56:24.744598 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237794    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 16:56:24.744598 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237798    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 16:56:24.744598 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237801    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 16:56:24.744598 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237804    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 16:56:24.744598 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237806    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 16:56:24.744598 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237809    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 16:56:24.744598 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237812    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 16:56:24.744598 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237814    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 16:56:24.744598 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237817    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 16:56:24.744598 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237821    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 16:56:24.744598 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237823    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 16:56:24.744598 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237826    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 16:56:24.744598 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237829    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 16:56:24.745260 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237831    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 16:56:24.745260 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237834    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 16:56:24.745260 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237837    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 16:56:24.745260 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237839    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 16:56:24.745260 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237843    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 16:56:24.745260 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237847    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 16:56:24.745260 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237850    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 16:56:24.745260 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.237853    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 16:56:24.745260 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.238450    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 16:56:24.745260 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.248818    2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 15 16:56:24.745260 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.248838    2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 15 16:56:24.745260 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.248860    2575 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 15 16:56:24.745260 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.248868    2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 15 16:56:24.745260 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.248929    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 16:56:24.745704 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.248936    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 16:56:24.745704 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.248940    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 16:56:24.745704 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.248945    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 16:56:24.745704 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.248948    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 16:56:24.745704 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.248952    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 16:56:24.745704 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.248954    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 16:56:24.745704 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.248957    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 16:56:24.745704 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.248960    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 16:56:24.745704 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.248962    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 16:56:24.745704 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.248965    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 16:56:24.745704 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.248967    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 16:56:24.745704 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.248970    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 16:56:24.745704 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.248972    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 16:56:24.745704 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.248975    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 16:56:24.745704 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.248978    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 16:56:24.745704 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.248981    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 16:56:24.745704 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.248983    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 16:56:24.745704 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.248986    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 16:56:24.745704 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.248989    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 16:56:24.746288 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.248993    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 16:56:24.746288 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.248996    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 16:56:24.746288 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249000    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 16:56:24.746288 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249002    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 16:56:24.746288 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249005    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 16:56:24.746288 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249008    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 16:56:24.746288 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249013    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 16:56:24.746288 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249017    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 16:56:24.746288 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249021    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 16:56:24.746288 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249031    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 16:56:24.746288 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249034    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 16:56:24.746288 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249037    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 16:56:24.746288 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249039    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 16:56:24.746288 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249042    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 16:56:24.746288 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249044    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 16:56:24.746288 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249047    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 16:56:24.746288 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249049    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 16:56:24.746288 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249052    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 16:56:24.746288 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249054    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 16:56:24.746288 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249057    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 16:56:24.746959 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249061    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 16:56:24.746959 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249065    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 16:56:24.746959 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249068    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 16:56:24.746959 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249071    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 16:56:24.746959 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249073    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 16:56:24.746959 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249076    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 16:56:24.746959 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249078    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 16:56:24.746959 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249081    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 16:56:24.746959 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249084    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 16:56:24.746959 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249087    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 16:56:24.746959 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249090    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 16:56:24.746959 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249097    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 16:56:24.746959 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249102    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 16:56:24.746959 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249106    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 16:56:24.746959 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249108    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 16:56:24.746959 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249111    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 16:56:24.746959 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249113    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 16:56:24.746959 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249116    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 16:56:24.746959 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249119    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 16:56:24.747544 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249121    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 16:56:24.747544 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249123    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 16:56:24.747544 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249126    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 16:56:24.747544 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249134    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 16:56:24.747544 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249137    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 16:56:24.747544 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249139    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 16:56:24.747544 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249142    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 16:56:24.747544 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249145    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 16:56:24.747544 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249147    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 16:56:24.747544 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249150    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 16:56:24.747544 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249153    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 16:56:24.747544 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249155    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 16:56:24.747544 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249157    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 16:56:24.747544 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249160    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 16:56:24.747544 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249162    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 16:56:24.747544 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249167    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 16:56:24.747544 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249172    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 16:56:24.747544 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249175    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 16:56:24.747544 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249180    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 16:56:24.747544 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249184    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 16:56:24.747544 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249187    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 16:56:24.748151 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249189    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 16:56:24.748151 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249192    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 16:56:24.748151 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249194    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 16:56:24.748151 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249200    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 16:56:24.748151 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249204    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 16:56:24.748151 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249206    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 16:56:24.748151 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249209    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 16:56:24.748151 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249212    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 16:56:24.748151 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249215    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 16:56:24.748151 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249218    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 16:56:24.748151 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249221    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 16:56:24.748151 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249224    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 16:56:24.748151 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249227    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 16:56:24.748151 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249229    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 16:56:24.748151 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249232    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 16:56:24.748151 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249244    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 16:56:24.748151 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249247    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 16:56:24.748151 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249252    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 16:56:24.748151 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249256    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 16:56:24.748784 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.249264    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 16:56:24.748784 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249413    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 16:56:24.748784 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249421    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 16:56:24.748784 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249425    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 16:56:24.748784 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249429    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 16:56:24.748784 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249432    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 16:56:24.748784 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249435    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 16:56:24.748784 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249439    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 16:56:24.748784 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249442    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 16:56:24.748784 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249445    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 16:56:24.748784 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249448    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 16:56:24.748784 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249451    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 16:56:24.748784 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249454    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 16:56:24.748784 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249457    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 16:56:24.748784 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249459    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 16:56:24.749228 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249462    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 16:56:24.749228 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249465    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 16:56:24.749228 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249469    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 16:56:24.749228 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249472    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 16:56:24.749228 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249474    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 16:56:24.749228 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249477    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 16:56:24.749228 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249480    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 16:56:24.749228 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249482    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 16:56:24.749228 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249485    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 16:56:24.749228 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249488    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 16:56:24.749228 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249493    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 16:56:24.749228 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249497    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 16:56:24.749228 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249501    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 16:56:24.749228 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249505    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 16:56:24.749228 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249515    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 16:56:24.749228 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249518    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 16:56:24.749228 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249520    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 16:56:24.749228 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249523    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 16:56:24.749228 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249526    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 16:56:24.749228 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249528    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 16:56:24.749839 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249530    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 16:56:24.749839 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249535    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 16:56:24.749839 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249538    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 16:56:24.749839 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249541    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 16:56:24.749839 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249545    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 16:56:24.749839 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249548    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 16:56:24.749839 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249551    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 16:56:24.749839 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249569    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 16:56:24.749839 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249574    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 16:56:24.749839 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249578    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 16:56:24.749839 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249582    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 16:56:24.749839 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249586    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 16:56:24.749839 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249589    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 16:56:24.749839 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249592    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 16:56:24.749839 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249596    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 16:56:24.749839 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249599    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 16:56:24.749839 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249601    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 16:56:24.749839 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249604    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 16:56:24.749839 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249606    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 16:56:24.750742 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249609    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 16:56:24.750742 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249611    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 16:56:24.750742 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249614    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 16:56:24.750742 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249616    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 16:56:24.750742 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249619    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 16:56:24.750742 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249622    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 16:56:24.750742 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249624    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 16:56:24.750742 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249627    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 16:56:24.750742 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249637    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 16:56:24.750742 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249639    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 16:56:24.750742 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249642    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 16:56:24.750742 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249644    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 16:56:24.750742 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249648    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 16:56:24.750742 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249652    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 16:56:24.750742 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249656    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 16:56:24.750742 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249678    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 16:56:24.750742 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249683    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 16:56:24.750742 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249689    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 16:56:24.750742 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249692    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 16:56:25.025464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249695    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 16:56:25.025464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249698    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 16:56:25.025464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249701    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 16:56:25.025464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249703    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 16:56:25.025464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249706    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 16:56:25.025464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249709    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 16:56:25.025464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249712    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 16:56:25.025464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249714    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 16:56:25.025464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249719    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 16:56:25.025464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249723    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 16:56:25.025464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249726    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 16:56:25.025464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249730    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 16:56:25.025464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249735    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 16:56:25.025464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249739    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 16:56:25.025464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249743    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 16:56:25.025464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249746    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 16:56:25.025464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249749    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 16:56:25.025464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249751    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 16:56:25.025464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249754    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 16:56:25.025464 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249756    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 16:56:25.028254 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249759    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 16:56:25.028254 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249761    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 16:56:25.028254 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249770    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 16:56:25.028254 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249772    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 16:56:25.028254 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249775    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 16:56:25.028254 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249778    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 16:56:25.028254 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:24.249780    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 16:56:25.028254 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.249785    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 16:56:25.028254 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.249926    2575 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 15 16:56:25.028254 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.252125    2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 15 16:56:25.028254 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.253635    2575 server.go:1019] "Starting client certificate rotation"
Jul 15 16:56:25.028254 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.253768    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 16:56:25.028254 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.253802    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 16:56:25.028730 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.256103    2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 16:56:25.028730 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.263789    2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 16:56:25.028730 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.273121    2575 log.go:25] "Validated CRI v1 runtime API"
Jul 15 16:56:25.028730 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.275388    2575 log.go:25] "Validated CRI v1 image API"
Jul 15 16:56:25.028730 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.276807    2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 15 16:56:25.028730 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.279046    2575 fs.go:135] Filesystem UUIDs: map[00d0c2a0-bed9-4b9e-a50c-cdd9b0addb36:/dev/nvme0n1p4 1c04405f-afa9-495c-9504-2217e09fa190:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 15 16:56:25.028730 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.279060    2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 15 16:56:25.029067 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.284657    2575 manager.go:216] Machine: {Timestamp:2026-07-15 16:56:24.283391887 +0000 UTC m=+0.300804974 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100827 MemoryCapacity:33164505088 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29e454189622a59d5eec5b95cc9730 SystemUUID:ec29e454-1896-22a5-9d5e-ec5b95cc9730 BootID:12b24a26-e810-4ec5-ac3d-01b98d2e7273 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582254592 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2d:56:16:1e:cf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2d:56:16:1e:cf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:e9:6a:42:10:94 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164505088 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 15 16:56:25.029067 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.284755    2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 15 16:56:25.029067 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.284832    2575 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 15 16:56:25.029067 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.287907    2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 15 16:56:25.029067 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.287931    2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-51.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 15 16:56:25.029067 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.288706    2575 topology_manager.go:138] "Creating topology manager with none policy"
Jul 15 16:56:25.029067 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.288717    2575 container_manager_linux.go:309] "Creating device plugin manager"
Jul 15 16:56:25.029067 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.288724    2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 16:56:25.029067 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.289390    2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 16:56:25.029067 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.289997    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 16:56:25.029067 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.291107    2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 15 16:56:25.029067 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.295710    2575 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 16:56:25.029067 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.295828    2575 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 15 16:56:25.029067 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.295855    2575 kubelet.go:486] "Attempting to sync node with API server"
Jul 15 16:56:25.029067 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.295870    2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 15 16:56:25.029067 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.295881    2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 15 16:56:25.029067 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.295899    2575 kubelet.go:398] "Adding apiserver pod source"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.295908    2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.296410    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.296426    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.298409    2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.300703    2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.300731    2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.300884    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.300921    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.300936    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.300947    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.300962    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.300973    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.301591    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.301599    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.301606    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.301614    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.301628    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.301636    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.302300    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.302306    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.305758    2575 server.go:1268] "Started kubelet"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.305774    2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.306010    2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.306081    2575 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.309912    2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.311277    2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-51.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.311369    2575 server.go:310] "Adding debug handlers to kubelet server"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.312908    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vshtt"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:24.313588    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:24.313708    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-51.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:24.317998    2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:24.318066    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-51.ec2.internal.18c284f3a1061b78  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-51.ec2.internal,UID:ip-10-0-142-51.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-51.ec2.internal,},FirstTimestamp:2026-07-15 16:56:24.305589112 +0000 UTC m=+0.323002199,LastTimestamp:2026-07-15 16:56:24.305589112 +0000 UTC m=+0.323002199,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-51.ec2.internal,}"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.320393    2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.320400    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:24.320588    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-51.ec2.internal\" not found"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.321157    2575 factory.go:55] Registering systemd factory
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.321169    2575 factory.go:223] Registration of the systemd container factory successfully
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.321195    2575 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.321212    2575 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 15 16:56:25.029604 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.321198    2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.321349    2575 reconstruct.go:97] "Volume reconstruction finished"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.321360    2575 reconciler.go:29] "Reconciler: start to sync state"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.321367    2575 factory.go:153] Registering CRI-O factory
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.321376    2575 factory.go:223] Registration of the crio container factory successfully
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.321424    2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.321445    2575 factory.go:103] Registering Raw factory
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.321455    2575 manager.go:1196] Started watching for new ooms in manager
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.321792    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vshtt"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.322190    2575 manager.go:349] Starting recovery of all containers
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:24.326121    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.337435    2575 manager.go:354] Recovery completed
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.341869    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.343785    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-51.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.343808    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-51.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.343818    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-51.ec2.internal" event="NodeHasSufficientPID"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:24.344047    2575 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-142-51.ec2.internal\" not found" node="ip-10-0-142-51.ec2.internal"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.344309    2575 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.344318    2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.344332    2575 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.344379    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.345030    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-51.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.345052    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-51.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.345064    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-51.ec2.internal" event="NodeHasSufficientPID"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.346303    2575 policy_none.go:49] "None policy: Start"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.346319    2575 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.346329    2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.348223    2575 policy_none.go:47] "Start"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.387166    2575 manager.go:337] "Starting Device Plugin manager"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:24.387197    2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.387207    2575 server.go:87] "Starting device plugin registration server"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.387462    2575 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.387474    2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.387580    2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.391729    2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.391741    2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:24.391764    2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:24.391817    2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-51.ec2.internal\" not found"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.488605    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.489668    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-51.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.489700    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-51.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.489711    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-51.ec2.internal" event="NodeHasSufficientPID"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.489741    2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-51.ec2.internal"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.498127    2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-51.ec2.internal"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:24.498149    2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-51.ec2.internal\": node \"ip-10-0-142-51.ec2.internal\" not found"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:24.517406    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-51.ec2.internal\" not found"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.517759    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 15 16:56:25.030820 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.519084    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.519126    2575 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.519154    2575 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:24.519196    2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.521793    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.624264    2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-51.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-51.ec2.internal"]
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.624366    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:24.624294    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-51.ec2.internal\" not found"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.625558    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-51.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.625602    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-51.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.625613    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-51.ec2.internal" event="NodeHasSufficientPID"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.627967    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.628119    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-51.ec2.internal"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.628145    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.628808    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-51.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.628840    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-51.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.628852    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-51.ec2.internal" event="NodeHasSufficientPID"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.628866    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-51.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.628887    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-51.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.628899    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-51.ec2.internal" event="NodeHasSufficientPID"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.631206    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-51.ec2.internal"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.631231    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.631918    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-51.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.631948    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-51.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.631962    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-51.ec2.internal" event="NodeHasSufficientPID"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:24.657744    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-51.ec2.internal\" not found" node="ip-10-0-142-51.ec2.internal"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:24.664542    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-51.ec2.internal\" not found" node="ip-10-0-142-51.ec2.internal"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.724442    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c961b9ae386440b3b5a6271935039d1b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-51.ec2.internal\" (UID: \"c961b9ae386440b3b5a6271935039d1b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-51.ec2.internal"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:24.724443    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-51.ec2.internal\" not found"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.724464    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c961b9ae386440b3b5a6271935039d1b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-51.ec2.internal\" (UID: \"c961b9ae386440b3b5a6271935039d1b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-51.ec2.internal"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.724498    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ef7caf07f6dae51e575ab6538fd06a86-config\") pod \"kube-apiserver-proxy-ip-10-0-142-51.ec2.internal\" (UID: \"ef7caf07f6dae51e575ab6538fd06a86\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-51.ec2.internal"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:24.824641    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-51.ec2.internal\" not found"
Jul 15 16:56:25.032127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.824671    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c961b9ae386440b3b5a6271935039d1b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-51.ec2.internal\" (UID: \"c961b9ae386440b3b5a6271935039d1b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-51.ec2.internal"
Jul 15 16:56:25.033098 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.824698    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c961b9ae386440b3b5a6271935039d1b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-51.ec2.internal\" (UID: \"c961b9ae386440b3b5a6271935039d1b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-51.ec2.internal"
Jul 15 16:56:25.033098 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.824718    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c961b9ae386440b3b5a6271935039d1b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-51.ec2.internal\" (UID: \"c961b9ae386440b3b5a6271935039d1b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-51.ec2.internal"
Jul 15 16:56:25.033098 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.824739    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ef7caf07f6dae51e575ab6538fd06a86-config\") pod \"kube-apiserver-proxy-ip-10-0-142-51.ec2.internal\" (UID: \"ef7caf07f6dae51e575ab6538fd06a86\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-51.ec2.internal"
Jul 15 16:56:25.033098 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.824755    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c961b9ae386440b3b5a6271935039d1b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-51.ec2.internal\" (UID: \"c961b9ae386440b3b5a6271935039d1b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-51.ec2.internal"
Jul 15 16:56:25.033098 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.824775    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ef7caf07f6dae51e575ab6538fd06a86-config\") pod \"kube-apiserver-proxy-ip-10-0-142-51.ec2.internal\" (UID: \"ef7caf07f6dae51e575ab6538fd06a86\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-51.ec2.internal"
Jul 15 16:56:25.033098 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:24.925412    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-51.ec2.internal\" not found"
Jul 15 16:56:25.033098 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.962369    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-51.ec2.internal"
Jul 15 16:56:25.033098 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:24.969145    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-51.ec2.internal"
Jul 15 16:56:25.033098 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:25.026230    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-51.ec2.internal\" not found"
Jul 15 16:56:25.141057 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:25.126785    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-51.ec2.internal\" not found"
Jul 15 16:56:25.251784 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:25.227216    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-51.ec2.internal\" not found"
Jul 15 16:56:25.251784 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.239760    2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 16:56:25.251784 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.249884    2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 16:56:25.254356 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.252897    2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 15 16:56:25.254356 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.253022    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 16:56:25.254356 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.253021    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 16:56:25.254356 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.253035    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.296298    2575 apiserver.go:52] "Watching apiserver"
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.316141    2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.319012    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-bnz8n","kube-system/konnectivity-agent-vf8th","openshift-dns/node-resolver-jdrr4","openshift-image-registry/node-ca-2brrq","openshift-multus/multus-additional-cni-plugins-mnvjv","openshift-network-operator/iptables-alerter-dtctc","openshift-ovn-kubernetes/ovnkube-node-kw2pl","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s","openshift-cluster-node-tuning-operator/tuned-ccfjt","openshift-multus/multus-fxndg","openshift-multus/network-metrics-daemon-7kv7g","openshift-network-diagnostics/network-check-target-9cncx"]
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.321210    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-51.ec2.internal"
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.321219    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.321375    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:25.321462    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bnz8n" podUID="ae13fea8-2c11-48da-b201-4007b9b6272c"
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.323255    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vf8th"
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.325444    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jdrr4"
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.325594    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2stph\""
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.325651    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.326664    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 16:51:25 +0000 UTC" deadline="2028-01-22 12:29:11.91628482 +0000 UTC"
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.326695    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13339h32m46.589599178s"
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.326726    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.328501    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ae13fea8-2c11-48da-b201-4007b9b6272c-kubelet-config\") pod \"global-pull-secret-syncer-bnz8n\" (UID: \"ae13fea8-2c11-48da-b201-4007b9b6272c\") " pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.328525    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2brrq"
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.328549    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ae13fea8-2c11-48da-b201-4007b9b6272c-dbus\") pod \"global-pull-secret-syncer-bnz8n\" (UID: \"ae13fea8-2c11-48da-b201-4007b9b6272c\") " pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.328587    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret\") pod \"global-pull-secret-syncer-bnz8n\" (UID: \"ae13fea8-2c11-48da-b201-4007b9b6272c\") " pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.328621    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/73f51511-7a5f-46dc-b252-9d05fdcefb15-agent-certs\") pod \"konnectivity-agent-vf8th\" (UID: \"73f51511-7a5f-46dc-b252-9d05fdcefb15\") " pod="kube-system/konnectivity-agent-vf8th"
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.328669    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/73f51511-7a5f-46dc-b252-9d05fdcefb15-konnectivity-ca\") pod \"konnectivity-agent-vf8th\" (UID: \"73f51511-7a5f-46dc-b252-9d05fdcefb15\") " pod="kube-system/konnectivity-agent-vf8th"
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.329442    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.329691    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.330067    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8xhf5\""
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.331133    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.331836    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.332124    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.332345    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zdhrx\""
Jul 15 16:56:25.370155 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.332420    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.333342    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.333465    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-51.ec2.internal"
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.334270    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dtctc"
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.334373    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.334422    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.334484    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.334775    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.334838    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.334943    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qkkz6\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.335776    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.336389    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-727ht\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.336534    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.337293    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.337478    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.337593    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.338987    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.339394    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.339598    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.339607    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.340456    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.340462    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.340470    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.340489    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pdknh\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.341266    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.341347    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.341472    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.341825    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.342100    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9p622\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.343207    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.343782    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.344139    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.344188    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.344250    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-66hnv\""
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.346113    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:25.371346 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:25.346181    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7kv7g" podUID="a62f9005-2c66-4d30-9102-121aae529be8"
Jul 15 16:56:25.372518 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.346245    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 15 16:56:25.372518 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.346489    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-g9xvl\""
Jul 15 16:56:25.372518 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.348242    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-51.ec2.internal"]
Jul 15 16:56:25.372518 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.348276    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-51.ec2.internal"]
Jul 15 16:56:25.372518 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.348344    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:25.372518 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:25.348396    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9cncx" podUID="8db7e638-ee27-4f89-a329-d6bb2fa60022"
Jul 15 16:56:25.372518 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.367068    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mbl88"
Jul 15 16:56:25.376085 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.373867    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mbl88"
Jul 15 16:56:25.486864 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.422774    2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 15 16:56:25.486864 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429313    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1de2bfc-4272-4761-b27a-76d5c4f9949e-tmp-dir\") pod \"node-resolver-jdrr4\" (UID: \"c1de2bfc-4272-4761-b27a-76d5c4f9949e\") " pod="openshift-dns/node-resolver-jdrr4"
Jul 15 16:56:25.486864 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429337    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-etc-sysctl-conf\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.486864 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429359    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5d97b3f4-a52b-4bd8-bf1a-ab72f5304ea4-serviceca\") pod \"node-ca-2brrq\" (UID: \"5d97b3f4-a52b-4bd8-bf1a-ab72f5304ea4\") " pod="openshift-image-registry/node-ca-2brrq"
Jul 15 16:56:25.486864 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429377    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-run-ovn\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.486864 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429393    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7f665db3-7e3b-44f1-968f-04413fe45670-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dhm8s\" (UID: \"7f665db3-7e3b-44f1-968f-04413fe45670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:25.486864 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429413    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-cnibin\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.486864 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429471    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-host-var-lib-cni-multus\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.486864 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429503    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-etc-modprobe-d\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.486864 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429539    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-etc-kubernetes\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.486864 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429593    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-etc-systemd\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.486864 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429631    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w52j\" (UniqueName: \"kubernetes.io/projected/4773188f-6494-428b-aa1c-b3256cc2b2f7-kube-api-access-6w52j\") pod \"iptables-alerter-dtctc\" (UID: \"4773188f-6494-428b-aa1c-b3256cc2b2f7\") " pod="openshift-network-operator/iptables-alerter-dtctc"
Jul 15 16:56:25.486864 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429649    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-systemd-units\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.486864 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429671    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-env-overrides\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.486864 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429699    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-system-cni-dir\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.486864 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429722    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs\") pod \"network-metrics-daemon-7kv7g\" (UID: \"a62f9005-2c66-4d30-9102-121aae529be8\") " pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:25.486864 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429744    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46nj9\" (UniqueName: \"kubernetes.io/projected/adca27dc-3730-43de-9940-070f0bf616f9-kube-api-access-46nj9\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.488433 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429764    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-lib-modules\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.488433 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429780    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7f665db3-7e3b-44f1-968f-04413fe45670-registration-dir\") pod \"aws-ebs-csi-driver-node-dhm8s\" (UID: \"7f665db3-7e3b-44f1-968f-04413fe45670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:25.488433 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429799    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-etc-kubernetes\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.488433 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429836    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/73f51511-7a5f-46dc-b252-9d05fdcefb15-agent-certs\") pod \"konnectivity-agent-vf8th\" (UID: \"73f51511-7a5f-46dc-b252-9d05fdcefb15\") " pod="kube-system/konnectivity-agent-vf8th"
Jul 15 16:56:25.488433 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429852    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/adca27dc-3730-43de-9940-070f0bf616f9-system-cni-dir\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.488433 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429873    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/adca27dc-3730-43de-9940-070f0bf616f9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.488433 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429892    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-etc-sysconfig\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.488433 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429913    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-host-var-lib-cni-bin\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.488433 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429934    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-host-run-ovn-kubernetes\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.488433 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429960    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-ovn-node-metrics-cert\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.488433 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.429984    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7f665db3-7e3b-44f1-968f-04413fe45670-socket-dir\") pod \"aws-ebs-csi-driver-node-dhm8s\" (UID: \"7f665db3-7e3b-44f1-968f-04413fe45670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:25.488433 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430005    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-os-release\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.488433 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430033    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-host-run-k8s-cni-cncf-io\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.488433 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430048    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/adca27dc-3730-43de-9940-070f0bf616f9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.488433 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430069    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/adca27dc-3730-43de-9940-070f0bf616f9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.488433 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430088    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-sys\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.489222 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430108    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvp52\" (UniqueName: \"kubernetes.io/projected/7f665db3-7e3b-44f1-968f-04413fe45670-kube-api-access-jvp52\") pod \"aws-ebs-csi-driver-node-dhm8s\" (UID: \"7f665db3-7e3b-44f1-968f-04413fe45670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:25.489222 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430123    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-hostroot\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.489222 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430142    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-run-openvswitch\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.489222 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430195    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-host-run-multus-certs\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.489222 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430220    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-run\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.489222 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430235    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-host-cni-bin\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.489222 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430272    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.489222 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430342    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-ovnkube-script-lib\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.489222 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430380    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-host-var-lib-kubelet\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.489222 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430414    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v72cz\" (UniqueName: \"kubernetes.io/projected/a62f9005-2c66-4d30-9102-121aae529be8-kube-api-access-v72cz\") pod \"network-metrics-daemon-7kv7g\" (UID: \"a62f9005-2c66-4d30-9102-121aae529be8\") " pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:25.489222 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430438    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/adca27dc-3730-43de-9940-070f0bf616f9-cni-binary-copy\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.489222 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430483    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/38685520-a6e8-4e97-bb11-63fcc732e3ba-cni-binary-copy\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.489222 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430504    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-host-run-netns\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.489222 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430529    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lvr6\" (UniqueName: \"kubernetes.io/projected/38685520-a6e8-4e97-bb11-63fcc732e3ba-kube-api-access-8lvr6\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.489222 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430553    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ae13fea8-2c11-48da-b201-4007b9b6272c-kubelet-config\") pod \"global-pull-secret-syncer-bnz8n\" (UID: \"ae13fea8-2c11-48da-b201-4007b9b6272c\") " pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:25.489222 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430600    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ae13fea8-2c11-48da-b201-4007b9b6272c-dbus\") pod \"global-pull-secret-syncer-bnz8n\" (UID: \"ae13fea8-2c11-48da-b201-4007b9b6272c\") " pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:25.490821 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430631    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/73f51511-7a5f-46dc-b252-9d05fdcefb15-konnectivity-ca\") pod \"konnectivity-agent-vf8th\" (UID: \"73f51511-7a5f-46dc-b252-9d05fdcefb15\") " pod="kube-system/konnectivity-agent-vf8th"
Jul 15 16:56:25.490821 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430656    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-var-lib-openvswitch\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.490821 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430664    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ae13fea8-2c11-48da-b201-4007b9b6272c-kubelet-config\") pod \"global-pull-secret-syncer-bnz8n\" (UID: \"ae13fea8-2c11-48da-b201-4007b9b6272c\") " pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:25.490821 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430686    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-log-socket\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.490821 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430723    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zn8s\" (UniqueName: \"kubernetes.io/projected/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-kube-api-access-5zn8s\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.490821 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430747    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7f665db3-7e3b-44f1-968f-04413fe45670-sys-fs\") pod \"aws-ebs-csi-driver-node-dhm8s\" (UID: \"7f665db3-7e3b-44f1-968f-04413fe45670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:25.490821 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430791    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-multus-socket-dir-parent\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.490821 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430809    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/adca27dc-3730-43de-9940-070f0bf616f9-cnibin\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.490821 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430818    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ae13fea8-2c11-48da-b201-4007b9b6272c-dbus\") pod \"global-pull-secret-syncer-bnz8n\" (UID: \"ae13fea8-2c11-48da-b201-4007b9b6272c\") " pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:25.490821 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430830    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-host\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.490821 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430852    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-etc-tuned\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.490821 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430884    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-node-log\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.490821 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430902    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7f665db3-7e3b-44f1-968f-04413fe45670-etc-selinux\") pod \"aws-ebs-csi-driver-node-dhm8s\" (UID: \"7f665db3-7e3b-44f1-968f-04413fe45670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:25.490821 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430945    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-multus-cni-dir\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.490821 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430966    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-var-lib-kubelet\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.490821 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.430981    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccjfr\" (UniqueName: \"kubernetes.io/projected/c1de2bfc-4272-4761-b27a-76d5c4f9949e-kube-api-access-ccjfr\") pod \"node-resolver-jdrr4\" (UID: \"c1de2bfc-4272-4761-b27a-76d5c4f9949e\") " pod="openshift-dns/node-resolver-jdrr4"
Jul 15 16:56:25.490821 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.431009    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp8nn\" (UniqueName: \"kubernetes.io/projected/5d97b3f4-a52b-4bd8-bf1a-ab72f5304ea4-kube-api-access-cp8nn\") pod \"node-ca-2brrq\" (UID: \"5d97b3f4-a52b-4bd8-bf1a-ab72f5304ea4\") " pod="openshift-image-registry/node-ca-2brrq"
Jul 15 16:56:25.491599 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.431044    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-ovnkube-config\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.491599 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.431090    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-multus-conf-dir\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.491599 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.431137    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret\") pod \"global-pull-secret-syncer-bnz8n\" (UID: \"ae13fea8-2c11-48da-b201-4007b9b6272c\") " pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:25.491599 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.431173    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/adca27dc-3730-43de-9940-070f0bf616f9-os-release\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.491599 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:25.431199    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 16:56:25.491599 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.431252    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tntpn\" (UniqueName: \"kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn\") pod \"network-check-target-9cncx\" (UID: \"8db7e638-ee27-4f89-a329-d6bb2fa60022\") " pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:25.491599 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.431311    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4773188f-6494-428b-aa1c-b3256cc2b2f7-iptables-alerter-script\") pod \"iptables-alerter-dtctc\" (UID: \"4773188f-6494-428b-aa1c-b3256cc2b2f7\") " pod="openshift-network-operator/iptables-alerter-dtctc"
Jul 15 16:56:25.491599 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.431343    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4773188f-6494-428b-aa1c-b3256cc2b2f7-host-slash\") pod \"iptables-alerter-dtctc\" (UID: \"4773188f-6494-428b-aa1c-b3256cc2b2f7\") " pod="openshift-network-operator/iptables-alerter-dtctc"
Jul 15 16:56:25.491599 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:25.431361    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret podName:ae13fea8-2c11-48da-b201-4007b9b6272c nodeName:}" failed. No retries permitted until 2026-07-15 16:56:25.931337369 +0000 UTC m=+1.948750465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret") pod "global-pull-secret-syncer-bnz8n" (UID: "ae13fea8-2c11-48da-b201-4007b9b6272c") : object "kube-system"/"original-pull-secret" not registered
Jul 15 16:56:25.491599 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.431390    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-host-kubelet\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.491599 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.431415    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-host-run-netns\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.491599 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.431439    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7f665db3-7e3b-44f1-968f-04413fe45670-device-dir\") pod \"aws-ebs-csi-driver-node-dhm8s\" (UID: \"7f665db3-7e3b-44f1-968f-04413fe45670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:25.491599 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.431466    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-etc-sysctl-d\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.491599 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.431490    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxb7c\" (UniqueName: \"kubernetes.io/projected/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-kube-api-access-zxb7c\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.491599 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.431512    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-etc-openvswitch\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.491599 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.431552    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/38685520-a6e8-4e97-bb11-63fcc732e3ba-multus-daemon-config\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.493997 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.431582    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c1de2bfc-4272-4761-b27a-76d5c4f9949e-hosts-file\") pod \"node-resolver-jdrr4\" (UID: \"c1de2bfc-4272-4761-b27a-76d5c4f9949e\") " pod="openshift-dns/node-resolver-jdrr4"
Jul 15 16:56:25.493997 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.431617    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-tmp\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.493997 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.431632    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d97b3f4-a52b-4bd8-bf1a-ab72f5304ea4-host\") pod \"node-ca-2brrq\" (UID: \"5d97b3f4-a52b-4bd8-bf1a-ab72f5304ea4\") " pod="openshift-image-registry/node-ca-2brrq"
Jul 15 16:56:25.493997 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.431678    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-host-slash\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.493997 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.431703    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-run-systemd\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.493997 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.431725    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-host-cni-netd\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.493997 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.433982    2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 15 16:56:25.493997 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.434037    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/73f51511-7a5f-46dc-b252-9d05fdcefb15-konnectivity-ca\") pod \"konnectivity-agent-vf8th\" (UID: \"73f51511-7a5f-46dc-b252-9d05fdcefb15\") " pod="kube-system/konnectivity-agent-vf8th"
Jul 15 16:56:25.493997 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.438428    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/73f51511-7a5f-46dc-b252-9d05fdcefb15-agent-certs\") pod \"konnectivity-agent-vf8th\" (UID: \"73f51511-7a5f-46dc-b252-9d05fdcefb15\") " pod="kube-system/konnectivity-agent-vf8th"
Jul 15 16:56:25.610001 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.531886    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxb7c\" (UniqueName: \"kubernetes.io/projected/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-kube-api-access-zxb7c\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.610001 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.531919    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-etc-openvswitch\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.610001 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.531963    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-etc-openvswitch\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.610001 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.531990    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/38685520-a6e8-4e97-bb11-63fcc732e3ba-multus-daemon-config\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.610001 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532009    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c1de2bfc-4272-4761-b27a-76d5c4f9949e-hosts-file\") pod \"node-resolver-jdrr4\" (UID: \"c1de2bfc-4272-4761-b27a-76d5c4f9949e\") " pod="openshift-dns/node-resolver-jdrr4"
Jul 15 16:56:25.610001 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532026    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-tmp\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.610001 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532103    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c1de2bfc-4272-4761-b27a-76d5c4f9949e-hosts-file\") pod \"node-resolver-jdrr4\" (UID: \"c1de2bfc-4272-4761-b27a-76d5c4f9949e\") " pod="openshift-dns/node-resolver-jdrr4"
Jul 15 16:56:25.610001 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532186    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d97b3f4-a52b-4bd8-bf1a-ab72f5304ea4-host\") pod \"node-ca-2brrq\" (UID: \"5d97b3f4-a52b-4bd8-bf1a-ab72f5304ea4\") " pod="openshift-image-registry/node-ca-2brrq"
Jul 15 16:56:25.610001 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532220    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-host-slash\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.610001 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532247    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-run-systemd\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.610001 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532272    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-host-cni-netd\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.610001 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532283    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d97b3f4-a52b-4bd8-bf1a-ab72f5304ea4-host\") pod \"node-ca-2brrq\" (UID: \"5d97b3f4-a52b-4bd8-bf1a-ab72f5304ea4\") " pod="openshift-image-registry/node-ca-2brrq"
Jul 15 16:56:25.610001 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532300    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1de2bfc-4272-4761-b27a-76d5c4f9949e-tmp-dir\") pod \"node-resolver-jdrr4\" (UID: \"c1de2bfc-4272-4761-b27a-76d5c4f9949e\") " pod="openshift-dns/node-resolver-jdrr4"
Jul 15 16:56:25.610001 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532345    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-host-slash\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.610001 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532362    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-host-cni-netd\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.610001 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532324    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-etc-sysctl-conf\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.610001 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532362    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-run-systemd\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.610001 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532401    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5d97b3f4-a52b-4bd8-bf1a-ab72f5304ea4-serviceca\") pod \"node-ca-2brrq\" (UID: \"5d97b3f4-a52b-4bd8-bf1a-ab72f5304ea4\") " pod="openshift-image-registry/node-ca-2brrq"
Jul 15 16:56:25.610805 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532445    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-run-ovn\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.610805 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532471    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7f665db3-7e3b-44f1-968f-04413fe45670-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dhm8s\" (UID: \"7f665db3-7e3b-44f1-968f-04413fe45670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:25.610805 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532488    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-etc-sysctl-conf\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.610805 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532499    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-cnibin\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.610805 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532507    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-run-ovn\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.610805 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532525    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-host-var-lib-cni-multus\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.610805 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532552    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-etc-modprobe-d\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.610805 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532566    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-cnibin\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.610805 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532593    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-etc-kubernetes\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.610805 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532599    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-host-var-lib-cni-multus\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.610805 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532603    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7f665db3-7e3b-44f1-968f-04413fe45670-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dhm8s\" (UID: \"7f665db3-7e3b-44f1-968f-04413fe45670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:25.610805 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532615    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-etc-systemd\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.610805 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532654    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w52j\" (UniqueName: \"kubernetes.io/projected/4773188f-6494-428b-aa1c-b3256cc2b2f7-kube-api-access-6w52j\") pod \"iptables-alerter-dtctc\" (UID: \"4773188f-6494-428b-aa1c-b3256cc2b2f7\") " pod="openshift-network-operator/iptables-alerter-dtctc"
Jul 15 16:56:25.610805 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532661    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-etc-systemd\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.610805 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532673    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-etc-modprobe-d\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.610805 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532661    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-etc-kubernetes\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.610805 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532792    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-systemd-units\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.610805 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532821    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-env-overrides\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.612374 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532855    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-system-cni-dir\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.612374 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532932    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs\") pod \"network-metrics-daemon-7kv7g\" (UID: \"a62f9005-2c66-4d30-9102-121aae529be8\") " pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:25.612374 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532951    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-system-cni-dir\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.612374 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532974    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46nj9\" (UniqueName: \"kubernetes.io/projected/adca27dc-3730-43de-9940-070f0bf616f9-kube-api-access-46nj9\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.612374 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.532864    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-systemd-units\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.612374 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533000    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-lib-modules\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.612374 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533104    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-lib-modules\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.612374 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533126    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7f665db3-7e3b-44f1-968f-04413fe45670-registration-dir\") pod \"aws-ebs-csi-driver-node-dhm8s\" (UID: \"7f665db3-7e3b-44f1-968f-04413fe45670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:25.612374 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:25.533148    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:56:25.612374 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533155    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-etc-kubernetes\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.612374 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533193    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/adca27dc-3730-43de-9940-070f0bf616f9-system-cni-dir\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.612374 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:25.533228    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs podName:a62f9005-2c66-4d30-9102-121aae529be8 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:26.033195057 +0000 UTC m=+2.050608133 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs") pod "network-metrics-daemon-7kv7g" (UID: "a62f9005-2c66-4d30-9102-121aae529be8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:56:25.612374 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533234    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/adca27dc-3730-43de-9940-070f0bf616f9-system-cni-dir\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.612374 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533228    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7f665db3-7e3b-44f1-968f-04413fe45670-registration-dir\") pod \"aws-ebs-csi-driver-node-dhm8s\" (UID: \"7f665db3-7e3b-44f1-968f-04413fe45670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:25.612374 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533231    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-etc-kubernetes\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.612374 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533274    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/adca27dc-3730-43de-9940-070f0bf616f9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.612374 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533311    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-etc-sysconfig\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.613151 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533364    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-host-var-lib-cni-bin\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.613151 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533417    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-host-run-ovn-kubernetes\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.613151 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533438    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-etc-sysconfig\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.613151 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533456    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-ovn-node-metrics-cert\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.613151 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533487    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-host-run-ovn-kubernetes\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.613151 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533487    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7f665db3-7e3b-44f1-968f-04413fe45670-socket-dir\") pod \"aws-ebs-csi-driver-node-dhm8s\" (UID: \"7f665db3-7e3b-44f1-968f-04413fe45670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:25.613151 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533527    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-os-release\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.613151 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533455    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-host-var-lib-cni-bin\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.613151 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533582    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-host-run-k8s-cni-cncf-io\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.613151 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533615    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7f665db3-7e3b-44f1-968f-04413fe45670-socket-dir\") pod \"aws-ebs-csi-driver-node-dhm8s\" (UID: \"7f665db3-7e3b-44f1-968f-04413fe45670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:25.613151 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533617    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/adca27dc-3730-43de-9940-070f0bf616f9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.613151 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533679    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-os-release\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.613151 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533620    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-host-run-k8s-cni-cncf-io\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.613151 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533695    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/adca27dc-3730-43de-9940-070f0bf616f9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.613151 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533753    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/adca27dc-3730-43de-9940-070f0bf616f9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.613151 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533815    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-sys\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.613151 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533849    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvp52\" (UniqueName: \"kubernetes.io/projected/7f665db3-7e3b-44f1-968f-04413fe45670-kube-api-access-jvp52\") pod \"aws-ebs-csi-driver-node-dhm8s\" (UID: \"7f665db3-7e3b-44f1-968f-04413fe45670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:25.613960 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533876    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-sys\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.613960 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533889    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-hostroot\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.613960 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533921    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-run-openvswitch\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.613960 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533939    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-hostroot\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.613960 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533953    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-host-run-multus-certs\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.613960 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533973    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-run-openvswitch\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.613960 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.533985    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-run\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.613960 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534005    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-host-run-multus-certs\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.613960 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534025    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-host-cni-bin\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.613960 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534038    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-run\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.613960 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534062    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.613960 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534102    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-host-cni-bin\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.613960 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534103    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-ovnkube-script-lib\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.613960 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534142    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-host-var-lib-kubelet\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.613960 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534170    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v72cz\" (UniqueName: \"kubernetes.io/projected/a62f9005-2c66-4d30-9102-121aae529be8-kube-api-access-v72cz\") pod \"network-metrics-daemon-7kv7g\" (UID: \"a62f9005-2c66-4d30-9102-121aae529be8\") " pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:25.613960 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534361    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-host-var-lib-kubelet\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.613960 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534367    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.615127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534387    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/adca27dc-3730-43de-9940-070f0bf616f9-cni-binary-copy\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.615127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534430    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/38685520-a6e8-4e97-bb11-63fcc732e3ba-cni-binary-copy\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.615127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534455    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-host-run-netns\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.615127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534494    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-host-run-netns\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.615127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534542    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lvr6\" (UniqueName: \"kubernetes.io/projected/38685520-a6e8-4e97-bb11-63fcc732e3ba-kube-api-access-8lvr6\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.615127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534619    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-var-lib-openvswitch\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.615127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534646    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-log-socket\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.615127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534671    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zn8s\" (UniqueName: \"kubernetes.io/projected/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-kube-api-access-5zn8s\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.615127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534691    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/38685520-a6e8-4e97-bb11-63fcc732e3ba-multus-daemon-config\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.615127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534696    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7f665db3-7e3b-44f1-968f-04413fe45670-sys-fs\") pod \"aws-ebs-csi-driver-node-dhm8s\" (UID: \"7f665db3-7e3b-44f1-968f-04413fe45670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:25.615127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534743    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-multus-socket-dir-parent\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.615127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534749    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7f665db3-7e3b-44f1-968f-04413fe45670-sys-fs\") pod \"aws-ebs-csi-driver-node-dhm8s\" (UID: \"7f665db3-7e3b-44f1-968f-04413fe45670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:25.615127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534751    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1de2bfc-4272-4761-b27a-76d5c4f9949e-tmp-dir\") pod \"node-resolver-jdrr4\" (UID: \"c1de2bfc-4272-4761-b27a-76d5c4f9949e\") " pod="openshift-dns/node-resolver-jdrr4"
Jul 15 16:56:25.615127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534770    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/adca27dc-3730-43de-9940-070f0bf616f9-cnibin\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.615127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534803    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-host\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.615127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534822    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-var-lib-openvswitch\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.615127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534829    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-etc-tuned\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.615127 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534856    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-node-log\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.615919 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534875    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-log-socket\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.615919 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534884    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7f665db3-7e3b-44f1-968f-04413fe45670-etc-selinux\") pod \"aws-ebs-csi-driver-node-dhm8s\" (UID: \"7f665db3-7e3b-44f1-968f-04413fe45670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:25.615919 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534912    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-multus-cni-dir\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.615919 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534938    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-var-lib-kubelet\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.615919 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534956    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5d97b3f4-a52b-4bd8-bf1a-ab72f5304ea4-serviceca\") pod \"node-ca-2brrq\" (UID: \"5d97b3f4-a52b-4bd8-bf1a-ab72f5304ea4\") " pod="openshift-image-registry/node-ca-2brrq"
Jul 15 16:56:25.615919 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534964    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccjfr\" (UniqueName: \"kubernetes.io/projected/c1de2bfc-4272-4761-b27a-76d5c4f9949e-kube-api-access-ccjfr\") pod \"node-resolver-jdrr4\" (UID: \"c1de2bfc-4272-4761-b27a-76d5c4f9949e\") " pod="openshift-dns/node-resolver-jdrr4"
Jul 15 16:56:25.615919 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.534991    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp8nn\" (UniqueName: \"kubernetes.io/projected/5d97b3f4-a52b-4bd8-bf1a-ab72f5304ea4-kube-api-access-cp8nn\") pod \"node-ca-2brrq\" (UID: \"5d97b3f4-a52b-4bd8-bf1a-ab72f5304ea4\") " pod="openshift-image-registry/node-ca-2brrq"
Jul 15 16:56:25.615919 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535022    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-multus-socket-dir-parent\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.615919 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535043    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/38685520-a6e8-4e97-bb11-63fcc732e3ba-cni-binary-copy\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.615919 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535067    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-ovnkube-config\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.615919 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535105    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-multus-conf-dir\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.615919 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535110    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-var-lib-kubelet\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.615919 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535125    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7f665db3-7e3b-44f1-968f-04413fe45670-etc-selinux\") pod \"aws-ebs-csi-driver-node-dhm8s\" (UID: \"7f665db3-7e3b-44f1-968f-04413fe45670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:25.615919 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535187    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-multus-cni-dir\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.615919 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535186    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/adca27dc-3730-43de-9940-070f0bf616f9-os-release\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.615919 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535213    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-host\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.615919 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535222    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tntpn\" (UniqueName: \"kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn\") pod \"network-check-target-9cncx\" (UID: \"8db7e638-ee27-4f89-a329-d6bb2fa60022\") " pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:25.615919 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535237    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/adca27dc-3730-43de-9940-070f0bf616f9-os-release\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.617083 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535252    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4773188f-6494-428b-aa1c-b3256cc2b2f7-iptables-alerter-script\") pod \"iptables-alerter-dtctc\" (UID: \"4773188f-6494-428b-aa1c-b3256cc2b2f7\") " pod="openshift-network-operator/iptables-alerter-dtctc"
Jul 15 16:56:25.617083 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535045    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/adca27dc-3730-43de-9940-070f0bf616f9-cnibin\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.617083 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535267    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-node-log\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.617083 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535282    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4773188f-6494-428b-aa1c-b3256cc2b2f7-host-slash\") pod \"iptables-alerter-dtctc\" (UID: \"4773188f-6494-428b-aa1c-b3256cc2b2f7\") " pod="openshift-network-operator/iptables-alerter-dtctc"
Jul 15 16:56:25.617083 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535318    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4773188f-6494-428b-aa1c-b3256cc2b2f7-host-slash\") pod \"iptables-alerter-dtctc\" (UID: \"4773188f-6494-428b-aa1c-b3256cc2b2f7\") " pod="openshift-network-operator/iptables-alerter-dtctc"
Jul 15 16:56:25.617083 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535363    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/38685520-a6e8-4e97-bb11-63fcc732e3ba-multus-conf-dir\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.617083 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535414    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-host-kubelet\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.617083 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535442    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-host-run-netns\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.617083 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535453    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-host-kubelet\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.617083 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535468    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7f665db3-7e3b-44f1-968f-04413fe45670-device-dir\") pod \"aws-ebs-csi-driver-node-dhm8s\" (UID: \"7f665db3-7e3b-44f1-968f-04413fe45670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:25.617083 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535493    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-etc-sysctl-d\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.617083 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535619    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-etc-sysctl-d\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.617083 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535688    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-tmp\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.617083 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535712    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-host-run-netns\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.617083 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535985    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/adca27dc-3730-43de-9940-070f0bf616f9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.617083 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.535999    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/adca27dc-3730-43de-9940-070f0bf616f9-cni-binary-copy\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.617083 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.536049    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-ovnkube-config\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.617873 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.536051    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7f665db3-7e3b-44f1-968f-04413fe45670-device-dir\") pod \"aws-ebs-csi-driver-node-dhm8s\" (UID: \"7f665db3-7e3b-44f1-968f-04413fe45670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:25.617873 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.536082    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4773188f-6494-428b-aa1c-b3256cc2b2f7-iptables-alerter-script\") pod \"iptables-alerter-dtctc\" (UID: \"4773188f-6494-428b-aa1c-b3256cc2b2f7\") " pod="openshift-network-operator/iptables-alerter-dtctc"
Jul 15 16:56:25.617873 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.536251    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-ovnkube-script-lib\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.617873 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.536392    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/adca27dc-3730-43de-9940-070f0bf616f9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.617873 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.536432    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-env-overrides\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.617873 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.537779    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-etc-tuned\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.617873 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.538099    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-ovn-node-metrics-cert\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.617873 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:25.545214    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 16:56:25.617873 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:25.545239    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 16:56:25.617873 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:25.545252    2575 projected.go:196] Error preparing data for projected volume kube-api-access-tntpn for pod openshift-network-diagnostics/network-check-target-9cncx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:56:25.617873 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:25.545309    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn podName:8db7e638-ee27-4f89-a329-d6bb2fa60022 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:26.045292752 +0000 UTC m=+2.062705867 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tntpn" (UniqueName: "kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn") pod "network-check-target-9cncx" (UID: "8db7e638-ee27-4f89-a329-d6bb2fa60022") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:56:25.617873 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.545779    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxb7c\" (UniqueName: \"kubernetes.io/projected/6c8a1644-a7d5-40e9-98df-0e08fc62e54d-kube-api-access-zxb7c\") pod \"tuned-ccfjt\" (UID: \"6c8a1644-a7d5-40e9-98df-0e08fc62e54d\") " pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:25.617873 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.546432    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w52j\" (UniqueName: \"kubernetes.io/projected/4773188f-6494-428b-aa1c-b3256cc2b2f7-kube-api-access-6w52j\") pod \"iptables-alerter-dtctc\" (UID: \"4773188f-6494-428b-aa1c-b3256cc2b2f7\") " pod="openshift-network-operator/iptables-alerter-dtctc"
Jul 15 16:56:25.617873 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.546548    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46nj9\" (UniqueName: \"kubernetes.io/projected/adca27dc-3730-43de-9940-070f0bf616f9-kube-api-access-46nj9\") pod \"multus-additional-cni-plugins-mnvjv\" (UID: \"adca27dc-3730-43de-9940-070f0bf616f9\") " pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:25.617873 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.547105    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvp52\" (UniqueName: \"kubernetes.io/projected/7f665db3-7e3b-44f1-968f-04413fe45670-kube-api-access-jvp52\") pod \"aws-ebs-csi-driver-node-dhm8s\" (UID: \"7f665db3-7e3b-44f1-968f-04413fe45670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:25.617873 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.547166    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v72cz\" (UniqueName: \"kubernetes.io/projected/a62f9005-2c66-4d30-9102-121aae529be8-kube-api-access-v72cz\") pod \"network-metrics-daemon-7kv7g\" (UID: \"a62f9005-2c66-4d30-9102-121aae529be8\") " pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:25.617873 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.548051    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zn8s\" (UniqueName: \"kubernetes.io/projected/9b91b45e-8528-42a0-ab1c-ac0b13c96c40-kube-api-access-5zn8s\") pod \"ovnkube-node-kw2pl\" (UID: \"9b91b45e-8528-42a0-ab1c-ac0b13c96c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:25.619062 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.548061    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp8nn\" (UniqueName: \"kubernetes.io/projected/5d97b3f4-a52b-4bd8-bf1a-ab72f5304ea4-kube-api-access-cp8nn\") pod \"node-ca-2brrq\" (UID: \"5d97b3f4-a52b-4bd8-bf1a-ab72f5304ea4\") " pod="openshift-image-registry/node-ca-2brrq"
Jul 15 16:56:25.619062 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.548809    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lvr6\" (UniqueName: \"kubernetes.io/projected/38685520-a6e8-4e97-bb11-63fcc732e3ba-kube-api-access-8lvr6\") pod \"multus-fxndg\" (UID: \"38685520-a6e8-4e97-bb11-63fcc732e3ba\") " pod="openshift-multus/multus-fxndg"
Jul 15 16:56:25.619062 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.549279    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccjfr\" (UniqueName: \"kubernetes.io/projected/c1de2bfc-4272-4761-b27a-76d5c4f9949e-kube-api-access-ccjfr\") pod \"node-resolver-jdrr4\" (UID: \"c1de2bfc-4272-4761-b27a-76d5c4f9949e\") " pod="openshift-dns/node-resolver-jdrr4"
Jul 15 16:56:25.728480 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.691440    2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 16:56:25.973786 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:25.937554    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret\") pod \"global-pull-secret-syncer-bnz8n\" (UID: \"ae13fea8-2c11-48da-b201-4007b9b6272c\") " pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:25.973786 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:25.937726    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 16:56:25.973786 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:25.937782    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret podName:ae13fea8-2c11-48da-b201-4007b9b6272c nodeName:}" failed. No retries permitted until 2026-07-15 16:56:26.937767541 +0000 UTC m=+2.955180616 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret") pod "global-pull-secret-syncer-bnz8n" (UID: "ae13fea8-2c11-48da-b201-4007b9b6272c") : object "kube-system"/"original-pull-secret" not registered
Jul 15 16:56:26.153584 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.038861    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs\") pod \"network-metrics-daemon-7kv7g\" (UID: \"a62f9005-2c66-4d30-9102-121aae529be8\") " pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:26.153584 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:26.039035    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:56:26.153584 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:26.039105    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs podName:a62f9005-2c66-4d30-9102-121aae529be8 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:27.039089435 +0000 UTC m=+3.056502511 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs") pod "network-metrics-daemon-7kv7g" (UID: "a62f9005-2c66-4d30-9102-121aae529be8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:56:26.153584 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.074467    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vf8th"
Jul 15 16:56:26.153584 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.095540    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jdrr4"
Jul 15 16:56:26.153584 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.112215    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2brrq"
Jul 15 16:56:26.153584 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.125514    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mnvjv"
Jul 15 16:56:26.153584 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.140004    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tntpn\" (UniqueName: \"kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn\") pod \"network-check-target-9cncx\" (UID: \"8db7e638-ee27-4f89-a329-d6bb2fa60022\") " pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:26.153584 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:26.140149    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 16:56:26.153584 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:26.140170    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 16:56:26.153584 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:26.140183    2575 projected.go:196] Error preparing data for projected volume kube-api-access-tntpn for pod openshift-network-diagnostics/network-check-target-9cncx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:56:26.153584 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:26.140242    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn podName:8db7e638-ee27-4f89-a329-d6bb2fa60022 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:27.140219807 +0000 UTC m=+3.157632891 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tntpn" (UniqueName: "kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn") pod "network-check-target-9cncx" (UID: "8db7e638-ee27-4f89-a329-d6bb2fa60022") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:56:26.345185 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.345161    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dtctc"
Jul 15 16:56:26.360312 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.360282    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:26.367410 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.367392    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s"
Jul 15 16:56:26.377507 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.377467    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 16:51:26 +0000 UTC" deadline="2028-04-21 15:44:13.090363676 +0000 UTC"
Jul 15 16:56:26.377507 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.377505    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15502h47m46.71286076s"
Jul 15 16:56:26.383380 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.383362    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ccfjt"
Jul 15 16:56:26.388364 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.388347    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fxndg"
Jul 15 16:56:26.443507 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:26.443466    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38685520_a6e8_4e97_bb11_63fcc732e3ba.slice/crio-cb07ab64ef19a609fd2cbdd8f5379c93a97fec1dedb9d46f19e856c4383bbd87 WatchSource:0}: Error finding container cb07ab64ef19a609fd2cbdd8f5379c93a97fec1dedb9d46f19e856c4383bbd87: Status 404 returned error can't find the container with id cb07ab64ef19a609fd2cbdd8f5379c93a97fec1dedb9d46f19e856c4383bbd87
Jul 15 16:56:26.444250 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:26.444228    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc961b9ae386440b3b5a6271935039d1b.slice/crio-6d647f8f0f7663fad4784a0029aea44ca58cc5f2f0090171e962a52342df8316 WatchSource:0}: Error finding container 6d647f8f0f7663fad4784a0029aea44ca58cc5f2f0090171e962a52342df8316: Status 404 returned error can't find the container with id 6d647f8f0f7663fad4784a0029aea44ca58cc5f2f0090171e962a52342df8316
Jul 15 16:56:26.444955 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:26.444932    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef7caf07f6dae51e575ab6538fd06a86.slice/crio-dab7bfe076d89c1da96c00e40edf540fe0c0b0859b9e93f046fd73cef3620a6f WatchSource:0}: Error finding container dab7bfe076d89c1da96c00e40edf540fe0c0b0859b9e93f046fd73cef3620a6f: Status 404 returned error can't find the container with id dab7bfe076d89c1da96c00e40edf540fe0c0b0859b9e93f046fd73cef3620a6f
Jul 15 16:56:26.448483 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.448467    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 16:56:26.506007 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:26.505980    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadca27dc_3730_43de_9940_070f0bf616f9.slice/crio-ef62a346c9083f222f4242286543ec248b95cfd623403e2bdfaf87f4facdceca WatchSource:0}: Error finding container ef62a346c9083f222f4242286543ec248b95cfd623403e2bdfaf87f4facdceca: Status 404 returned error can't find the container with id ef62a346c9083f222f4242286543ec248b95cfd623403e2bdfaf87f4facdceca
Jul 15 16:56:26.522739 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.522719    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:26.522739 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.522729    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:26.522918 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:26.522813    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9cncx" podUID="8db7e638-ee27-4f89-a329-d6bb2fa60022"
Jul 15 16:56:26.522966 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:26.522906    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bnz8n" podUID="ae13fea8-2c11-48da-b201-4007b9b6272c"
Jul 15 16:56:26.524264 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.524229    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mnvjv" event={"ID":"adca27dc-3730-43de-9940-070f0bf616f9","Type":"ContainerStarted","Data":"ef62a346c9083f222f4242286543ec248b95cfd623403e2bdfaf87f4facdceca"}
Jul 15 16:56:26.525380 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.525350    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-51.ec2.internal" event={"ID":"ef7caf07f6dae51e575ab6538fd06a86","Type":"ContainerStarted","Data":"dab7bfe076d89c1da96c00e40edf540fe0c0b0859b9e93f046fd73cef3620a6f"}
Jul 15 16:56:26.526399 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.526379    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-51.ec2.internal" event={"ID":"c961b9ae386440b3b5a6271935039d1b","Type":"ContainerStarted","Data":"6d647f8f0f7663fad4784a0029aea44ca58cc5f2f0090171e962a52342df8316"}
Jul 15 16:56:26.527312 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.527292    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fxndg" event={"ID":"38685520-a6e8-4e97-bb11-63fcc732e3ba","Type":"ContainerStarted","Data":"cb07ab64ef19a609fd2cbdd8f5379c93a97fec1dedb9d46f19e856c4383bbd87"}
Jul 15 16:56:26.632964 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:26.632943    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73f51511_7a5f_46dc_b252_9d05fdcefb15.slice/crio-6379d756af6a02f774217a3b89cf7bdd6d083d42a9b21215b78e0f7f1d8f85ac WatchSource:0}: Error finding container 6379d756af6a02f774217a3b89cf7bdd6d083d42a9b21215b78e0f7f1d8f85ac: Status 404 returned error can't find the container with id 6379d756af6a02f774217a3b89cf7bdd6d083d42a9b21215b78e0f7f1d8f85ac
Jul 15 16:56:26.697688 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:26.697659    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4773188f_6494_428b_aa1c_b3256cc2b2f7.slice/crio-d3fa05a7224a8f0e9ab24ff8364f1fe233f19bb962f26999d3ff74ab35f5b5c8 WatchSource:0}: Error finding container d3fa05a7224a8f0e9ab24ff8364f1fe233f19bb962f26999d3ff74ab35f5b5c8: Status 404 returned error can't find the container with id d3fa05a7224a8f0e9ab24ff8364f1fe233f19bb962f26999d3ff74ab35f5b5c8
Jul 15 16:56:26.703911 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.703887    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 16:56:26.733461 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:26.733435    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c8a1644_a7d5_40e9_98df_0e08fc62e54d.slice/crio-6b40a7e2de8608302ecfa5b17ee77dcc92324f51d8246208d0f805e5313154a4 WatchSource:0}: Error finding container 6b40a7e2de8608302ecfa5b17ee77dcc92324f51d8246208d0f805e5313154a4: Status 404 returned error can't find the container with id 6b40a7e2de8608302ecfa5b17ee77dcc92324f51d8246208d0f805e5313154a4
Jul 15 16:56:26.882648 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:26.882613    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f665db3_7e3b_44f1_968f_04413fe45670.slice/crio-ca814372ad64a8a41f80ef8bb8435fdd0aa97e3c0eb2df7f55e01ff7938a3946 WatchSource:0}: Error finding container ca814372ad64a8a41f80ef8bb8435fdd0aa97e3c0eb2df7f55e01ff7938a3946: Status 404 returned error can't find the container with id ca814372ad64a8a41f80ef8bb8435fdd0aa97e3c0eb2df7f55e01ff7938a3946
Jul 15 16:56:26.901907 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:26.901874    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1de2bfc_4272_4761_b27a_76d5c4f9949e.slice/crio-4fa38595e1b978c6680471c712a6f58036dfa998d207d22cc43a87a7bcd927fc WatchSource:0}: Error finding container 4fa38595e1b978c6680471c712a6f58036dfa998d207d22cc43a87a7bcd927fc: Status 404 returned error can't find the container with id 4fa38595e1b978c6680471c712a6f58036dfa998d207d22cc43a87a7bcd927fc
Jul 15 16:56:26.924007 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:26.923981    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b91b45e_8528_42a0_ab1c_ac0b13c96c40.slice/crio-400c7b84f7d127411f7e3f260ff95f3f63ba230e7d830f7a1bb73b7aefd708f3 WatchSource:0}: Error finding container 400c7b84f7d127411f7e3f260ff95f3f63ba230e7d830f7a1bb73b7aefd708f3: Status 404 returned error can't find the container with id 400c7b84f7d127411f7e3f260ff95f3f63ba230e7d830f7a1bb73b7aefd708f3
Jul 15 16:56:26.937649 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.937612    2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 16:56:26.946557 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:26.946495    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret\") pod \"global-pull-secret-syncer-bnz8n\" (UID: \"ae13fea8-2c11-48da-b201-4007b9b6272c\") " pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:26.946681 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:26.946650    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 16:56:26.946750 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:26.946727    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret podName:ae13fea8-2c11-48da-b201-4007b9b6272c nodeName:}" failed. No retries permitted until 2026-07-15 16:56:28.946704901 +0000 UTC m=+4.964117984 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret") pod "global-pull-secret-syncer-bnz8n" (UID: "ae13fea8-2c11-48da-b201-4007b9b6272c") : object "kube-system"/"original-pull-secret" not registered
Jul 15 16:56:27.029874 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:27.029844    2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 16:56:27.047447 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:27.047415    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs\") pod \"network-metrics-daemon-7kv7g\" (UID: \"a62f9005-2c66-4d30-9102-121aae529be8\") " pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:27.047663 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:27.047584    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:56:27.047663 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:27.047651    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs podName:a62f9005-2c66-4d30-9102-121aae529be8 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:29.047630017 +0000 UTC m=+5.065043099 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs") pod "network-metrics-daemon-7kv7g" (UID: "a62f9005-2c66-4d30-9102-121aae529be8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:56:27.148647 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:27.148604    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tntpn\" (UniqueName: \"kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn\") pod \"network-check-target-9cncx\" (UID: \"8db7e638-ee27-4f89-a329-d6bb2fa60022\") " pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:27.148843 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:27.148826    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 16:56:27.148922 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:27.148851    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 16:56:27.148922 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:27.148866    2575 projected.go:196] Error preparing data for projected volume kube-api-access-tntpn for pod openshift-network-diagnostics/network-check-target-9cncx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:56:27.149040 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:27.148931    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn podName:8db7e638-ee27-4f89-a329-d6bb2fa60022 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:29.148908834 +0000 UTC m=+5.166321932 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tntpn" (UniqueName: "kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn") pod "network-check-target-9cncx" (UID: "8db7e638-ee27-4f89-a329-d6bb2fa60022") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:56:27.378578 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:27.378502    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 16:51:26 +0000 UTC" deadline="2027-12-21 08:20:00.685476394 +0000 UTC"
Jul 15 16:56:27.379247 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:27.378588    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12567h23m33.306893692s"
Jul 15 16:56:27.519868 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:27.519829    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:27.520088 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:27.519988    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7kv7g" podUID="a62f9005-2c66-4d30-9102-121aae529be8"
Jul 15 16:56:27.564130 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:27.564016    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jdrr4" event={"ID":"c1de2bfc-4272-4761-b27a-76d5c4f9949e","Type":"ContainerStarted","Data":"4fa38595e1b978c6680471c712a6f58036dfa998d207d22cc43a87a7bcd927fc"}
Jul 15 16:56:27.568532 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:27.568501    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ccfjt" event={"ID":"6c8a1644-a7d5-40e9-98df-0e08fc62e54d","Type":"ContainerStarted","Data":"6b40a7e2de8608302ecfa5b17ee77dcc92324f51d8246208d0f805e5313154a4"}
Jul 15 16:56:27.593475 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:27.593375    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s" event={"ID":"7f665db3-7e3b-44f1-968f-04413fe45670","Type":"ContainerStarted","Data":"ca814372ad64a8a41f80ef8bb8435fdd0aa97e3c0eb2df7f55e01ff7938a3946"}
Jul 15 16:56:27.605426 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:27.605124    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dtctc" event={"ID":"4773188f-6494-428b-aa1c-b3256cc2b2f7","Type":"ContainerStarted","Data":"d3fa05a7224a8f0e9ab24ff8364f1fe233f19bb962f26999d3ff74ab35f5b5c8"}
Jul 15 16:56:27.624585 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:27.624541    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vf8th" event={"ID":"73f51511-7a5f-46dc-b252-9d05fdcefb15","Type":"ContainerStarted","Data":"6379d756af6a02f774217a3b89cf7bdd6d083d42a9b21215b78e0f7f1d8f85ac"}
Jul 15 16:56:27.638836 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:27.638758    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl" event={"ID":"9b91b45e-8528-42a0-ab1c-ac0b13c96c40","Type":"ContainerStarted","Data":"400c7b84f7d127411f7e3f260ff95f3f63ba230e7d830f7a1bb73b7aefd708f3"}
Jul 15 16:56:28.522170 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:28.522134    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:28.522692 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:28.522288    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bnz8n" podUID="ae13fea8-2c11-48da-b201-4007b9b6272c"
Jul 15 16:56:28.522692 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:28.522373    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:28.522692 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:28.522452    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9cncx" podUID="8db7e638-ee27-4f89-a329-d6bb2fa60022"
Jul 15 16:56:28.968078 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:28.968041    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret\") pod \"global-pull-secret-syncer-bnz8n\" (UID: \"ae13fea8-2c11-48da-b201-4007b9b6272c\") " pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:28.968288 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:28.968247    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 16:56:28.968367 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:28.968320    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret podName:ae13fea8-2c11-48da-b201-4007b9b6272c nodeName:}" failed. No retries permitted until 2026-07-15 16:56:32.968301486 +0000 UTC m=+8.985714567 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret") pod "global-pull-secret-syncer-bnz8n" (UID: "ae13fea8-2c11-48da-b201-4007b9b6272c") : object "kube-system"/"original-pull-secret" not registered
Jul 15 16:56:29.068865 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:29.068824    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs\") pod \"network-metrics-daemon-7kv7g\" (UID: \"a62f9005-2c66-4d30-9102-121aae529be8\") " pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:29.069069 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:29.069038    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:56:29.069137 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:29.069118    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs podName:a62f9005-2c66-4d30-9102-121aae529be8 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:33.069088162 +0000 UTC m=+9.086501241 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs") pod "network-metrics-daemon-7kv7g" (UID: "a62f9005-2c66-4d30-9102-121aae529be8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:56:29.169385 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:29.169344    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tntpn\" (UniqueName: \"kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn\") pod \"network-check-target-9cncx\" (UID: \"8db7e638-ee27-4f89-a329-d6bb2fa60022\") " pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:29.169691 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:29.169594    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 16:56:29.169691 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:29.169637    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 16:56:29.169691 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:29.169652    2575 projected.go:196] Error preparing data for projected volume kube-api-access-tntpn for pod openshift-network-diagnostics/network-check-target-9cncx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:56:29.169914 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:29.169732    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn podName:8db7e638-ee27-4f89-a329-d6bb2fa60022 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:33.169708877 +0000 UTC m=+9.187121954 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tntpn" (UniqueName: "kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn") pod "network-check-target-9cncx" (UID: "8db7e638-ee27-4f89-a329-d6bb2fa60022") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:56:29.520157 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:29.520124    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:29.520363 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:29.520282    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7kv7g" podUID="a62f9005-2c66-4d30-9102-121aae529be8"
Jul 15 16:56:30.520408 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:30.520372    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:30.520915 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:30.520536    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9cncx" podUID="8db7e638-ee27-4f89-a329-d6bb2fa60022"
Jul 15 16:56:30.520915 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:30.520409    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:30.521041 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:30.520949    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bnz8n" podUID="ae13fea8-2c11-48da-b201-4007b9b6272c"
Jul 15 16:56:31.520176 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:31.519680    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:31.520176 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:31.519838    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7kv7g" podUID="a62f9005-2c66-4d30-9102-121aae529be8"
Jul 15 16:56:32.521018 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:32.520351    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:32.521018 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:32.520369    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:32.521018 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:32.520488    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bnz8n" podUID="ae13fea8-2c11-48da-b201-4007b9b6272c"
Jul 15 16:56:32.521018 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:32.520651    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9cncx" podUID="8db7e638-ee27-4f89-a329-d6bb2fa60022"
Jul 15 16:56:33.004514 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:33.004398    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret\") pod \"global-pull-secret-syncer-bnz8n\" (UID: \"ae13fea8-2c11-48da-b201-4007b9b6272c\") " pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:33.004811 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:33.004612    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 16:56:33.004811 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:33.004685    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret podName:ae13fea8-2c11-48da-b201-4007b9b6272c nodeName:}" failed. No retries permitted until 2026-07-15 16:56:41.004662557 +0000 UTC m=+17.022075653 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret") pod "global-pull-secret-syncer-bnz8n" (UID: "ae13fea8-2c11-48da-b201-4007b9b6272c") : object "kube-system"/"original-pull-secret" not registered
Jul 15 16:56:33.105393 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:33.105357    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs\") pod \"network-metrics-daemon-7kv7g\" (UID: \"a62f9005-2c66-4d30-9102-121aae529be8\") " pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:33.105612 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:33.105505    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:56:33.105612 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:33.105566    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs podName:a62f9005-2c66-4d30-9102-121aae529be8 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:41.105545935 +0000 UTC m=+17.122959013 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs") pod "network-metrics-daemon-7kv7g" (UID: "a62f9005-2c66-4d30-9102-121aae529be8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:56:33.206168 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:33.205990    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tntpn\" (UniqueName: \"kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn\") pod \"network-check-target-9cncx\" (UID: \"8db7e638-ee27-4f89-a329-d6bb2fa60022\") " pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:33.206364 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:33.206190    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 16:56:33.206364 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:33.206209    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 16:56:33.206364 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:33.206222    2575 projected.go:196] Error preparing data for projected volume kube-api-access-tntpn for pod openshift-network-diagnostics/network-check-target-9cncx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:56:33.206364 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:33.206282    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn podName:8db7e638-ee27-4f89-a329-d6bb2fa60022 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:41.20626345 +0000 UTC m=+17.223676528 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tntpn" (UniqueName: "kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn") pod "network-check-target-9cncx" (UID: "8db7e638-ee27-4f89-a329-d6bb2fa60022") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:56:33.519420 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:33.519386    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:33.519640 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:33.519575    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7kv7g" podUID="a62f9005-2c66-4d30-9102-121aae529be8"
Jul 15 16:56:34.522221 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:34.522053    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:34.522221 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:34.522100    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:34.522221 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:34.522169    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bnz8n" podUID="ae13fea8-2c11-48da-b201-4007b9b6272c"
Jul 15 16:56:34.522871 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:34.522575    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9cncx" podUID="8db7e638-ee27-4f89-a329-d6bb2fa60022"
Jul 15 16:56:35.520581 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:35.520234    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:35.520581 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:35.520388    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7kv7g" podUID="a62f9005-2c66-4d30-9102-121aae529be8"
Jul 15 16:56:36.520388 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:36.520125    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:36.520889 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:36.520497    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bnz8n" podUID="ae13fea8-2c11-48da-b201-4007b9b6272c"
Jul 15 16:56:36.520889 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:36.520178    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:36.520991 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:36.520967    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9cncx" podUID="8db7e638-ee27-4f89-a329-d6bb2fa60022"
Jul 15 16:56:37.519952 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:37.519917    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:37.520117 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:37.520032    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7kv7g" podUID="a62f9005-2c66-4d30-9102-121aae529be8"
Jul 15 16:56:38.520312 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:38.520286    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:38.520610 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:38.520295    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:38.520610 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:38.520376    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9cncx" podUID="8db7e638-ee27-4f89-a329-d6bb2fa60022"
Jul 15 16:56:38.520610 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:38.520447    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bnz8n" podUID="ae13fea8-2c11-48da-b201-4007b9b6272c"
Jul 15 16:56:39.519902 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:39.519875    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:39.520094 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:39.519991    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7kv7g" podUID="a62f9005-2c66-4d30-9102-121aae529be8"
Jul 15 16:56:40.519447 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:40.519417    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:40.519920 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:40.519542    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bnz8n" podUID="ae13fea8-2c11-48da-b201-4007b9b6272c"
Jul 15 16:56:40.519920 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:40.519615    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:40.519920 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:40.519710    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9cncx" podUID="8db7e638-ee27-4f89-a329-d6bb2fa60022"
Jul 15 16:56:41.068645 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:41.068606    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret\") pod \"global-pull-secret-syncer-bnz8n\" (UID: \"ae13fea8-2c11-48da-b201-4007b9b6272c\") " pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:41.068806 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:41.068769    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 16:56:41.068864 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:41.068851    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret podName:ae13fea8-2c11-48da-b201-4007b9b6272c nodeName:}" failed. No retries permitted until 2026-07-15 16:56:57.068831565 +0000 UTC m=+33.086244645 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret") pod "global-pull-secret-syncer-bnz8n" (UID: "ae13fea8-2c11-48da-b201-4007b9b6272c") : object "kube-system"/"original-pull-secret" not registered
Jul 15 16:56:41.169122 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:41.169093    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs\") pod \"network-metrics-daemon-7kv7g\" (UID: \"a62f9005-2c66-4d30-9102-121aae529be8\") " pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:41.169293 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:41.169222    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:56:41.169293 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:41.169280    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs podName:a62f9005-2c66-4d30-9102-121aae529be8 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:57.169261796 +0000 UTC m=+33.186674870 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs") pod "network-metrics-daemon-7kv7g" (UID: "a62f9005-2c66-4d30-9102-121aae529be8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:56:41.270014 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:41.269980    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tntpn\" (UniqueName: \"kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn\") pod \"network-check-target-9cncx\" (UID: \"8db7e638-ee27-4f89-a329-d6bb2fa60022\") " pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:41.270199 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:41.270179    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 16:56:41.270255 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:41.270208    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 16:56:41.270255 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:41.270222    2575 projected.go:196] Error preparing data for projected volume kube-api-access-tntpn for pod openshift-network-diagnostics/network-check-target-9cncx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:56:41.270324 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:41.270284    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn podName:8db7e638-ee27-4f89-a329-d6bb2fa60022 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:57.270265262 +0000 UTC m=+33.287678353 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tntpn" (UniqueName: "kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn") pod "network-check-target-9cncx" (UID: "8db7e638-ee27-4f89-a329-d6bb2fa60022") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:56:41.520333 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:41.520302    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:41.520741 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:41.520421    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7kv7g" podUID="a62f9005-2c66-4d30-9102-121aae529be8"
Jul 15 16:56:42.520371 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:42.520337    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:42.520793 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:42.520457    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bnz8n" podUID="ae13fea8-2c11-48da-b201-4007b9b6272c"
Jul 15 16:56:42.520793 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:42.520507    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:42.520793 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:42.520617    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9cncx" podUID="8db7e638-ee27-4f89-a329-d6bb2fa60022"
Jul 15 16:56:43.519674 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:43.519641    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:43.519841 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:43.519755    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7kv7g" podUID="a62f9005-2c66-4d30-9102-121aae529be8"
Jul 15 16:56:44.522476 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:44.522442    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:44.522916 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:44.522545    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9cncx" podUID="8db7e638-ee27-4f89-a329-d6bb2fa60022"
Jul 15 16:56:44.522916 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:44.522594    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:44.522916 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:44.522711    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bnz8n" podUID="ae13fea8-2c11-48da-b201-4007b9b6272c"
Jul 15 16:56:45.519665 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:45.519633    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:45.519850 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:45.519758    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7kv7g" podUID="a62f9005-2c66-4d30-9102-121aae529be8"
Jul 15 16:56:46.522342 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:46.522317    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:46.522756 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:46.522404    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9cncx" podUID="8db7e638-ee27-4f89-a329-d6bb2fa60022"
Jul 15 16:56:46.522756 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:46.522461    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:46.522756 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:46.522578    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bnz8n" podUID="ae13fea8-2c11-48da-b201-4007b9b6272c"
Jul 15 16:56:47.520454 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.520246    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:47.520707 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:47.520572    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7kv7g" podUID="a62f9005-2c66-4d30-9102-121aae529be8"
Jul 15 16:56:47.694749 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.694710    2575 generic.go:356] "Generic (PLEG): container finished" podID="adca27dc-3730-43de-9940-070f0bf616f9" containerID="176b58d2ea65365437db815e5a4e90a2a69a5819fd818d9b5c2075262443cc54" exitCode=0
Jul 15 16:56:47.695647 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.694791    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mnvjv" event={"ID":"adca27dc-3730-43de-9940-070f0bf616f9","Type":"ContainerDied","Data":"176b58d2ea65365437db815e5a4e90a2a69a5819fd818d9b5c2075262443cc54"}
Jul 15 16:56:47.697529 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.697501    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-51.ec2.internal" event={"ID":"ef7caf07f6dae51e575ab6538fd06a86","Type":"ContainerStarted","Data":"91c3a0ebe7674a6cc01d8d42ea566bad55af928e90791563498f19e5fd08b3ca"}
Jul 15 16:56:47.700549 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.700045    2575 generic.go:356] "Generic (PLEG): container finished" podID="c961b9ae386440b3b5a6271935039d1b" containerID="2207dbdce634eda3216565054510efd736df0bdeff15a690d8daa311b1eea878" exitCode=0
Jul 15 16:56:47.700549 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.700117    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-51.ec2.internal" event={"ID":"c961b9ae386440b3b5a6271935039d1b","Type":"ContainerDied","Data":"2207dbdce634eda3216565054510efd736df0bdeff15a690d8daa311b1eea878"}
Jul 15 16:56:47.703370 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.703086    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fxndg" event={"ID":"38685520-a6e8-4e97-bb11-63fcc732e3ba","Type":"ContainerStarted","Data":"a33bd7204b0f953063f4bf001410d62edf0471808a2df0e07a53edc6cb461f69"}
Jul 15 16:56:47.705005 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.704979    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s" event={"ID":"7f665db3-7e3b-44f1-968f-04413fe45670","Type":"ContainerStarted","Data":"136be97ab2631986f9f3a9d7e2204af2314333eb5eaeaa9a194464ed5d7a9b25"}
Jul 15 16:56:47.706889 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.706861    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vf8th" event={"ID":"73f51511-7a5f-46dc-b252-9d05fdcefb15","Type":"ContainerStarted","Data":"c2ebbf1cce79362f38dc7f6319c715861b8bb4d2194235f4c663004ad8b85ecb"}
Jul 15 16:56:47.708289 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.708242    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2brrq" event={"ID":"5d97b3f4-a52b-4bd8-bf1a-ab72f5304ea4","Type":"ContainerStarted","Data":"9749356ba69475824ae213217be2538ece343994d6c603b3c897931ee3a52a0f"}
Jul 15 16:56:47.712720 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.712701    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 16:56:47.713638 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.713612    2575 generic.go:356] "Generic (PLEG): container finished" podID="9b91b45e-8528-42a0-ab1c-ac0b13c96c40" containerID="41743ff097584f6eb9cfa7fc355596c2fd7369d6ef86130a4301395b76e1a113" exitCode=1
Jul 15 16:56:47.713734 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.713682    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl" event={"ID":"9b91b45e-8528-42a0-ab1c-ac0b13c96c40","Type":"ContainerStarted","Data":"68013258096f3a0b747822bba9f517824ede7876a9a8ed6dee5ab1c8c2ce984c"}
Jul 15 16:56:47.713811 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.713748    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl" event={"ID":"9b91b45e-8528-42a0-ab1c-ac0b13c96c40","Type":"ContainerStarted","Data":"2a7c8b8bb79911003f00ba0a18944c144cffc94f3e925fc7cccf644632f9139f"}
Jul 15 16:56:47.713869 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.713843    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl" event={"ID":"9b91b45e-8528-42a0-ab1c-ac0b13c96c40","Type":"ContainerStarted","Data":"131a8deee4b7f8dd97762ac924115c6c09f7bb24dd9eb31d9358b88f8cb05d39"}
Jul 15 16:56:47.713923 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.713897    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl" event={"ID":"9b91b45e-8528-42a0-ab1c-ac0b13c96c40","Type":"ContainerStarted","Data":"1d9accc8a9185443b21c4df8d9590db30e01f5d40058afa3126a1e9a5c936a31"}
Jul 15 16:56:47.713979 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.713946    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl" event={"ID":"9b91b45e-8528-42a0-ab1c-ac0b13c96c40","Type":"ContainerDied","Data":"41743ff097584f6eb9cfa7fc355596c2fd7369d6ef86130a4301395b76e1a113"}
Jul 15 16:56:47.714032 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.713996    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl" event={"ID":"9b91b45e-8528-42a0-ab1c-ac0b13c96c40","Type":"ContainerStarted","Data":"2546e8a4251717eb6cfe3105842edb6f9d394a240589283ab251cc948ecc0777"}
Jul 15 16:56:47.717130 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.717090    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jdrr4" event={"ID":"c1de2bfc-4272-4761-b27a-76d5c4f9949e","Type":"ContainerStarted","Data":"d3fea94ebc79448c5c46cce2e575dbbe0d72beedb31d0ffb6575af27a7ba561a"}
Jul 15 16:56:47.719737 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.719315    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ccfjt" event={"ID":"6c8a1644-a7d5-40e9-98df-0e08fc62e54d","Type":"ContainerStarted","Data":"151a0994ea4c7320c982118c45ecbc4e029d78c1801f7d78a186856bad577e28"}
Jul 15 16:56:47.763077 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.763023    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-51.ec2.internal" podStartSLOduration=21.763005085 podStartE2EDuration="21.763005085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:47.762749225 +0000 UTC m=+23.780162321" watchObservedRunningTime="2026-07-15 16:56:47.763005085 +0000 UTC m=+23.780418182"
Jul 15 16:56:47.763220 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.763155    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jdrr4" podStartSLOduration=2.916516429 podStartE2EDuration="22.763149596s" totalImagesPullingTime="19.846633167s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:47.750438606 +0000 UTC m=+23.767851703" watchObservedRunningTime="2026-07-15 16:56:47.763149596 +0000 UTC m=+23.780562689"
Jul 15 16:56:47.777267 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.777225    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vf8th" podStartSLOduration=2.654950193 podStartE2EDuration="22.777211101s" totalImagesPullingTime="20.122260908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:47.776394572 +0000 UTC m=+23.793807688" watchObservedRunningTime="2026-07-15 16:56:47.777211101 +0000 UTC m=+23.794624196"
Jul 15 16:56:47.813089 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.813052    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ccfjt" podStartSLOduration=2.779728212 podStartE2EDuration="22.813033528s" totalImagesPullingTime="20.033305316s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:47.81241604 +0000 UTC m=+23.829829163" watchObservedRunningTime="2026-07-15 16:56:47.813033528 +0000 UTC m=+23.830446626"
Jul 15 16:56:47.813215 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:47.813167    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fxndg" podStartSLOduration=2.451813535 podStartE2EDuration="22.813161733s" totalImagesPullingTime="20.361348198s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:47.793995135 +0000 UTC m=+23.811408230" watchObservedRunningTime="2026-07-15 16:56:47.813161733 +0000 UTC m=+23.830574829"
Jul 15 16:56:48.519923 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:48.519887    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:48.520187 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:48.520010    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9cncx" podUID="8db7e638-ee27-4f89-a329-d6bb2fa60022"
Jul 15 16:56:48.520187 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:48.520056    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:48.520187 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:48.520143    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bnz8n" podUID="ae13fea8-2c11-48da-b201-4007b9b6272c"
Jul 15 16:56:48.722395 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:48.722361    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dtctc" event={"ID":"4773188f-6494-428b-aa1c-b3256cc2b2f7","Type":"ContainerStarted","Data":"c3876c88227658ce777ffba11ece6245b799424ba54a5bb41712c963ea52c9cd"}
Jul 15 16:56:48.724157 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:48.724127    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-51.ec2.internal" event={"ID":"c961b9ae386440b3b5a6271935039d1b","Type":"ContainerStarted","Data":"53beef37eaff16e3b1ea5b228fd081f467bf7aa48905d2dbc36a7f70b13fb0b7"}
Jul 15 16:56:48.743992 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:48.743956    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dtctc" podStartSLOduration=3.677873608 podStartE2EDuration="23.743943498s" totalImagesPullingTime="20.06606989s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:48.74379641 +0000 UTC m=+24.761209507" watchObservedRunningTime="2026-07-15 16:56:48.743943498 +0000 UTC m=+24.761356593"
Jul 15 16:56:48.757808 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:48.757765    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-51.ec2.internal" podStartSLOduration=22.75775282 podStartE2EDuration="22.75775282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:48.757544717 +0000 UTC m=+24.774957814" watchObservedRunningTime="2026-07-15 16:56:48.75775282 +0000 UTC m=+24.775165916"
Jul 15 16:56:48.994387 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:48.994337    2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 15 16:56:49.143530 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:49.143499    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vf8th"
Jul 15 16:56:49.144103 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:49.144079    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vf8th"
Jul 15 16:56:49.404766 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:49.404612    2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T16:56:48.994364881Z","UUID":"768d05f1-f509-49e0-b8b5-737e951a1aa3","Handler":null,"Name":"","Endpoint":""}
Jul 15 16:56:49.406526 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:49.406471    2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 15 16:56:49.406526 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:49.406519    2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 15 16:56:49.520001 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:49.519965    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:49.520161 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:49.520108    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7kv7g" podUID="a62f9005-2c66-4d30-9102-121aae529be8"
Jul 15 16:56:49.729486 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:49.729408    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s" event={"ID":"7f665db3-7e3b-44f1-968f-04413fe45670","Type":"ContainerStarted","Data":"74b729264bb150deb1b9339544174a8e33f09df36547631c8d0f6930438079d2"}
Jul 15 16:56:49.731200 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:49.731169    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2brrq" event={"ID":"5d97b3f4-a52b-4bd8-bf1a-ab72f5304ea4","Type":"ContainerStarted","Data":"8fa9c12123e4a5aebaed07461b6f86a6681c5c6038feda420ebb2f9b3eee2940"}
Jul 15 16:56:49.737842 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:49.735531    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 16:56:49.738118 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:49.738082    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl" event={"ID":"9b91b45e-8528-42a0-ab1c-ac0b13c96c40","Type":"ContainerStarted","Data":"ada8cca9338d05596ffadcab703bf3c493ea3c6cf05d4777efb433d25bef9d74"}
Jul 15 16:56:49.738497 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:49.738450    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vf8th"
Jul 15 16:56:49.739016 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:49.738977    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vf8th"
Jul 15 16:56:49.760309 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:49.760263    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2brrq" podStartSLOduration=22.655776085 podStartE2EDuration="24.760245011s" totalImagesPullingTime="2.104468926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:49.745229591 +0000 UTC m=+25.762642707" watchObservedRunningTime="2026-07-15 16:56:49.760245011 +0000 UTC m=+25.777658108"
Jul 15 16:56:50.520099 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:50.519868    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:50.520357 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:50.519868    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:50.520357 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:50.520196    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9cncx" podUID="8db7e638-ee27-4f89-a329-d6bb2fa60022"
Jul 15 16:56:50.520357 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:50.520273    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bnz8n" podUID="ae13fea8-2c11-48da-b201-4007b9b6272c"
Jul 15 16:56:50.742630 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:50.742596    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s" event={"ID":"7f665db3-7e3b-44f1-968f-04413fe45670","Type":"ContainerStarted","Data":"9456b6c57659529a8699aadc54f58e7bfa2a1083cb3b350f723727f4e01f8fee"}
Jul 15 16:56:51.520257 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:51.520226    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:51.520447 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:51.520350    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7kv7g" podUID="a62f9005-2c66-4d30-9102-121aae529be8"
Jul 15 16:56:52.520034 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:52.520010    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:52.520668 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:52.520117    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9cncx" podUID="8db7e638-ee27-4f89-a329-d6bb2fa60022"
Jul 15 16:56:52.520668 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:52.520168    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:52.520668 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:52.520254    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bnz8n" podUID="ae13fea8-2c11-48da-b201-4007b9b6272c"
Jul 15 16:56:52.751620 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:52.751400    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 16:56:52.751961 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:52.751936    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl" event={"ID":"9b91b45e-8528-42a0-ab1c-ac0b13c96c40","Type":"ContainerStarted","Data":"fb8932ba62bedc49127b53374ad25650499355aa1568819c673b40e98ac90894"}
Jul 15 16:56:52.752222 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:52.752196    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:52.752373 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:52.752360    2575 scope.go:117] "RemoveContainer" containerID="41743ff097584f6eb9cfa7fc355596c2fd7369d6ef86130a4301395b76e1a113"
Jul 15 16:56:52.768615 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:52.768596    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:52.778481 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:52.778446    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dhm8s" podStartSLOduration=5.109933966 podStartE2EDuration="27.778431473s" totalImagesPullingTime="22.668497507s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:50.765125866 +0000 UTC m=+26.782538961" watchObservedRunningTime="2026-07-15 16:56:52.778431473 +0000 UTC m=+28.795844572"
Jul 15 16:56:53.519372 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:53.519347    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:53.519485 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:53.519448    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7kv7g" podUID="a62f9005-2c66-4d30-9102-121aae529be8"
Jul 15 16:56:53.755331 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:53.755241    2575 generic.go:356] "Generic (PLEG): container finished" podID="adca27dc-3730-43de-9940-070f0bf616f9" containerID="b339999c789cec6237cdf3930f78e8f91c9fca59578a80f66ea5e5544bfe6244" exitCode=0
Jul 15 16:56:53.755914 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:53.755327    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mnvjv" event={"ID":"adca27dc-3730-43de-9940-070f0bf616f9","Type":"ContainerDied","Data":"b339999c789cec6237cdf3930f78e8f91c9fca59578a80f66ea5e5544bfe6244"}
Jul 15 16:56:53.759194 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:53.759179    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 16:56:53.759546 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:53.759523    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl" event={"ID":"9b91b45e-8528-42a0-ab1c-ac0b13c96c40","Type":"ContainerStarted","Data":"57cac8c1d00fea4bfe27a70ee1e6e50b8aab8a240b97d6d47099811f76fcd255"}
Jul 15 16:56:53.759829 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:53.759812    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:53.759916 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:53.759885    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:53.777092 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:53.777065    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:56:53.803676 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:53.803621    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl" podStartSLOduration=8.901214491 podStartE2EDuration="28.803606301s" totalImagesPullingTime="19.90239181s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:53.802219164 +0000 UTC m=+29.819632280" watchObservedRunningTime="2026-07-15 16:56:53.803606301 +0000 UTC m=+29.821019397"
Jul 15 16:56:54.520620 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:54.520594    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:54.520822 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:54.520699    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9cncx" podUID="8db7e638-ee27-4f89-a329-d6bb2fa60022"
Jul 15 16:56:54.520822 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:54.520738    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:54.520930 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:54.520832    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bnz8n" podUID="ae13fea8-2c11-48da-b201-4007b9b6272c"
Jul 15 16:56:54.663599 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:54.663543    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bnz8n"]
Jul 15 16:56:54.666847 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:54.666808    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9cncx"]
Jul 15 16:56:54.667495 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:54.667459    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7kv7g"]
Jul 15 16:56:54.667643 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:54.667588    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:54.667725 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:54.667700    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7kv7g" podUID="a62f9005-2c66-4d30-9102-121aae529be8"
Jul 15 16:56:54.761536 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:54.761474    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:54.761962 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:54.761479    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:54.761962 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:54.761588    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9cncx" podUID="8db7e638-ee27-4f89-a329-d6bb2fa60022"
Jul 15 16:56:54.761962 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:54.761641    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bnz8n" podUID="ae13fea8-2c11-48da-b201-4007b9b6272c"
Jul 15 16:56:55.766100 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:55.765902    2575 generic.go:356] "Generic (PLEG): container finished" podID="adca27dc-3730-43de-9940-070f0bf616f9" containerID="4b96f5a077d53d7af236315ffa3689a81dd459a58632a7783fc149823e26dd95" exitCode=0
Jul 15 16:56:55.766571 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:55.765947    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mnvjv" event={"ID":"adca27dc-3730-43de-9940-070f0bf616f9","Type":"ContainerDied","Data":"4b96f5a077d53d7af236315ffa3689a81dd459a58632a7783fc149823e26dd95"}
Jul 15 16:56:56.519434 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:56.519370    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:56.519597 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:56.519370    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:56.519597 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:56.519485    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7kv7g" podUID="a62f9005-2c66-4d30-9102-121aae529be8"
Jul 15 16:56:56.519597 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:56.519374    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:56.519597 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:56.519578    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bnz8n" podUID="ae13fea8-2c11-48da-b201-4007b9b6272c"
Jul 15 16:56:56.519749 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:56.519637    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9cncx" podUID="8db7e638-ee27-4f89-a329-d6bb2fa60022"
Jul 15 16:56:57.090102 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:57.090039    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret\") pod \"global-pull-secret-syncer-bnz8n\" (UID: \"ae13fea8-2c11-48da-b201-4007b9b6272c\") " pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:57.090402 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:57.090140    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 16:56:57.090402 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:57.090199    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret podName:ae13fea8-2c11-48da-b201-4007b9b6272c nodeName:}" failed. No retries permitted until 2026-07-15 16:57:29.090184877 +0000 UTC m=+65.107597951 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret") pod "global-pull-secret-syncer-bnz8n" (UID: "ae13fea8-2c11-48da-b201-4007b9b6272c") : object "kube-system"/"original-pull-secret" not registered
Jul 15 16:56:57.190297 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:57.190272    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs\") pod \"network-metrics-daemon-7kv7g\" (UID: \"a62f9005-2c66-4d30-9102-121aae529be8\") " pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:57.190417 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:57.190367    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:56:57.190417 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:57.190412    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs podName:a62f9005-2c66-4d30-9102-121aae529be8 nodeName:}" failed. No retries permitted until 2026-07-15 16:57:29.190398913 +0000 UTC m=+65.207811987 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs") pod "network-metrics-daemon-7kv7g" (UID: "a62f9005-2c66-4d30-9102-121aae529be8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:56:57.291032 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:57.291003    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tntpn\" (UniqueName: \"kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn\") pod \"network-check-target-9cncx\" (UID: \"8db7e638-ee27-4f89-a329-d6bb2fa60022\") " pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:57.291164 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:57.291117    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 16:56:57.291164 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:57.291132    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 16:56:57.291164 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:57.291140    2575 projected.go:196] Error preparing data for projected volume kube-api-access-tntpn for pod openshift-network-diagnostics/network-check-target-9cncx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:56:57.291263 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:57.291183    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn podName:8db7e638-ee27-4f89-a329-d6bb2fa60022 nodeName:}" failed. No retries permitted until 2026-07-15 16:57:29.291170357 +0000 UTC m=+65.308583444 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tntpn" (UniqueName: "kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn") pod "network-check-target-9cncx" (UID: "8db7e638-ee27-4f89-a329-d6bb2fa60022") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:56:57.772102 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:57.772070    2575 generic.go:356] "Generic (PLEG): container finished" podID="adca27dc-3730-43de-9940-070f0bf616f9" containerID="32b55b1b4221ea76142df3162bfb00d3f1fecb6235129a4bff9f95385c371ce2" exitCode=0
Jul 15 16:56:57.772239 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:57.772136    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mnvjv" event={"ID":"adca27dc-3730-43de-9940-070f0bf616f9","Type":"ContainerDied","Data":"32b55b1b4221ea76142df3162bfb00d3f1fecb6235129a4bff9f95385c371ce2"}
Jul 15 16:56:58.520187 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:58.520158    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:56:58.520708 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:58.520158    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:56:58.520708 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:58.520288    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:56:58.520708 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:58.520286    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bnz8n" podUID="ae13fea8-2c11-48da-b201-4007b9b6272c"
Jul 15 16:56:58.520708 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:58.520376    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9cncx" podUID="8db7e638-ee27-4f89-a329-d6bb2fa60022"
Jul 15 16:56:58.520708 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:58.520460    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7kv7g" podUID="a62f9005-2c66-4d30-9102-121aae529be8"
Jul 15 16:56:59.302730 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.302698    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-51.ec2.internal" event="NodeReady"
Jul 15 16:56:59.302898 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.302839    2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 15 16:56:59.372615 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.372578    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-f6q75"]
Jul 15 16:56:59.374618 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.374598    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f6q75"
Jul 15 16:56:59.376213 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.376188    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-p2qjz"]
Jul 15 16:56:59.377986 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.377965    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p2qjz"
Jul 15 16:56:59.383746 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.383286    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 15 16:56:59.383746 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.383315    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 15 16:56:59.384374 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.383820    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 15 16:56:59.384374 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.383869    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 15 16:56:59.387549 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.384986    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rf5xj\""
Jul 15 16:56:59.387549 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.385113    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 15 16:56:59.387549 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.385186    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lllhc\""
Jul 15 16:56:59.400121 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.400096    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p2qjz"]
Jul 15 16:56:59.400858 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.400837    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f6q75"]
Jul 15 16:56:59.506648 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.506541    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-tmp-dir\") pod \"dns-default-f6q75\" (UID: \"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3\") " pod="openshift-dns/dns-default-f6q75"
Jul 15 16:56:59.506648 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.506609    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00f13bdd-a991-489f-879a-019b72447eb5-cert\") pod \"ingress-canary-p2qjz\" (UID: \"00f13bdd-a991-489f-879a-019b72447eb5\") " pod="openshift-ingress-canary/ingress-canary-p2qjz"
Jul 15 16:56:59.506648 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.506639    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-config-volume\") pod \"dns-default-f6q75\" (UID: \"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3\") " pod="openshift-dns/dns-default-f6q75"
Jul 15 16:56:59.506889 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.506687    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls\") pod \"dns-default-f6q75\" (UID: \"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3\") " pod="openshift-dns/dns-default-f6q75"
Jul 15 16:56:59.506889 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.506719    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf7nr\" (UniqueName: \"kubernetes.io/projected/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-kube-api-access-hf7nr\") pod \"dns-default-f6q75\" (UID: \"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3\") " pod="openshift-dns/dns-default-f6q75"
Jul 15 16:56:59.506889 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.506769    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6cgq\" (UniqueName: \"kubernetes.io/projected/00f13bdd-a991-489f-879a-019b72447eb5-kube-api-access-x6cgq\") pod \"ingress-canary-p2qjz\" (UID: \"00f13bdd-a991-489f-879a-019b72447eb5\") " pod="openshift-ingress-canary/ingress-canary-p2qjz"
Jul 15 16:56:59.608085 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.608053    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf7nr\" (UniqueName: \"kubernetes.io/projected/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-kube-api-access-hf7nr\") pod \"dns-default-f6q75\" (UID: \"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3\") " pod="openshift-dns/dns-default-f6q75"
Jul 15 16:56:59.608650 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.608130    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6cgq\" (UniqueName: \"kubernetes.io/projected/00f13bdd-a991-489f-879a-019b72447eb5-kube-api-access-x6cgq\") pod \"ingress-canary-p2qjz\" (UID: \"00f13bdd-a991-489f-879a-019b72447eb5\") " pod="openshift-ingress-canary/ingress-canary-p2qjz"
Jul 15 16:56:59.608650 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.608170    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-tmp-dir\") pod \"dns-default-f6q75\" (UID: \"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3\") " pod="openshift-dns/dns-default-f6q75"
Jul 15 16:56:59.608650 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.608198    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00f13bdd-a991-489f-879a-019b72447eb5-cert\") pod \"ingress-canary-p2qjz\" (UID: \"00f13bdd-a991-489f-879a-019b72447eb5\") " pod="openshift-ingress-canary/ingress-canary-p2qjz"
Jul 15 16:56:59.608650 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.608231    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-config-volume\") pod \"dns-default-f6q75\" (UID: \"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3\") " pod="openshift-dns/dns-default-f6q75"
Jul 15 16:56:59.608650 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.608283    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls\") pod \"dns-default-f6q75\" (UID: \"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3\") " pod="openshift-dns/dns-default-f6q75"
Jul 15 16:56:59.608650 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:59.608387    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 16:56:59.608650 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:56:59.608455    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls podName:2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3 nodeName:}" failed. No retries permitted until 2026-07-15 16:57:00.108432635 +0000 UTC m=+36.125845725 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls") pod "dns-default-f6q75" (UID: "2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3") : secret "dns-default-metrics-tls" not found
Jul 15 16:56:59.608650 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.608546    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-tmp-dir\") pod \"dns-default-f6q75\" (UID: \"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3\") " pod="openshift-dns/dns-default-f6q75"
Jul 15 16:56:59.609046 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.608947    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-config-volume\") pod \"dns-default-f6q75\" (UID: \"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3\") " pod="openshift-dns/dns-default-f6q75"
Jul 15 16:56:59.613393 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.613371    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00f13bdd-a991-489f-879a-019b72447eb5-cert\") pod \"ingress-canary-p2qjz\" (UID: \"00f13bdd-a991-489f-879a-019b72447eb5\") " pod="openshift-ingress-canary/ingress-canary-p2qjz"
Jul 15 16:56:59.621721 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.621701    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6cgq\" (UniqueName: \"kubernetes.io/projected/00f13bdd-a991-489f-879a-019b72447eb5-kube-api-access-x6cgq\") pod \"ingress-canary-p2qjz\" (UID: \"00f13bdd-a991-489f-879a-019b72447eb5\") " pod="openshift-ingress-canary/ingress-canary-p2qjz"
Jul 15 16:56:59.627447 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.627427    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf7nr\" (UniqueName: \"kubernetes.io/projected/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-kube-api-access-hf7nr\") pod \"dns-default-f6q75\" (UID: \"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3\") " pod="openshift-dns/dns-default-f6q75"
Jul 15 16:56:59.694106 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.694078    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p2qjz"
Jul 15 16:56:59.845184 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:56:59.844998    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p2qjz"]
Jul 15 16:56:59.849271 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:56:59.849241    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00f13bdd_a991_489f_879a_019b72447eb5.slice/crio-8b8d6300892831d573a3c90935fb798dc23714e0c0e90c1b8a4735c886899317 WatchSource:0}: Error finding container 8b8d6300892831d573a3c90935fb798dc23714e0c0e90c1b8a4735c886899317: Status 404 returned error can't find the container with id 8b8d6300892831d573a3c90935fb798dc23714e0c0e90c1b8a4735c886899317
Jul 15 16:57:00.111618 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:00.111501    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls\") pod \"dns-default-f6q75\" (UID: \"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3\") " pod="openshift-dns/dns-default-f6q75"
Jul 15 16:57:00.111766 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:57:00.111653    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 16:57:00.111766 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:57:00.111720    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls podName:2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3 nodeName:}" failed. No retries permitted until 2026-07-15 16:57:01.111701604 +0000 UTC m=+37.129114744 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls") pod "dns-default-f6q75" (UID: "2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3") : secret "dns-default-metrics-tls" not found
Jul 15 16:57:00.520348 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:00.520280    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:57:00.520514 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:00.520293    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:57:00.520514 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:00.520420    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:57:00.523325 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:00.523304    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 16:57:00.523444 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:00.523304    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 16:57:00.523444 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:00.523364    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 16:57:00.523952 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:00.523933    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7h7tx\""
Jul 15 16:57:00.524041 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:00.524026    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 16:57:00.524108 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:00.524065    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pqk7d\""
Jul 15 16:57:00.780742 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:00.780653    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p2qjz" event={"ID":"00f13bdd-a991-489f-879a-019b72447eb5","Type":"ContainerStarted","Data":"8b8d6300892831d573a3c90935fb798dc23714e0c0e90c1b8a4735c886899317"}
Jul 15 16:57:01.118894 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:01.118860    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls\") pod \"dns-default-f6q75\" (UID: \"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3\") " pod="openshift-dns/dns-default-f6q75"
Jul 15 16:57:01.119091 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:57:01.119028    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 16:57:01.119163 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:57:01.119105    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls podName:2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3 nodeName:}" failed. No retries permitted until 2026-07-15 16:57:03.119082402 +0000 UTC m=+39.136495483 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls") pod "dns-default-f6q75" (UID: "2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3") : secret "dns-default-metrics-tls" not found
Jul 15 16:57:01.787196 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:01.787045    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mnvjv" event={"ID":"adca27dc-3730-43de-9940-070f0bf616f9","Type":"ContainerStarted","Data":"2843bf50ba8f140e31a56e8a077bfb173a841dd2c101cb34957e2083ea32d2c1"}
Jul 15 16:57:02.792501 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:02.792464    2575 generic.go:356] "Generic (PLEG): container finished" podID="adca27dc-3730-43de-9940-070f0bf616f9" containerID="2843bf50ba8f140e31a56e8a077bfb173a841dd2c101cb34957e2083ea32d2c1" exitCode=0
Jul 15 16:57:02.792977 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:02.792525    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mnvjv" event={"ID":"adca27dc-3730-43de-9940-070f0bf616f9","Type":"ContainerDied","Data":"2843bf50ba8f140e31a56e8a077bfb173a841dd2c101cb34957e2083ea32d2c1"}
Jul 15 16:57:03.135689 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:03.135646    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls\") pod \"dns-default-f6q75\" (UID: \"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3\") " pod="openshift-dns/dns-default-f6q75"
Jul 15 16:57:03.135872 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:57:03.135833    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 16:57:03.135953 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:57:03.135911    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls podName:2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3 nodeName:}" failed. No retries permitted until 2026-07-15 16:57:07.135894529 +0000 UTC m=+43.153307607 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls") pod "dns-default-f6q75" (UID: "2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3") : secret "dns-default-metrics-tls" not found
Jul 15 16:57:03.797749 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:03.797715    2575 generic.go:356] "Generic (PLEG): container finished" podID="adca27dc-3730-43de-9940-070f0bf616f9" containerID="e68e6fa84535abe342de25644126501492339073047f8b745287a4120aa3d513" exitCode=0
Jul 15 16:57:03.798116 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:03.797770    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mnvjv" event={"ID":"adca27dc-3730-43de-9940-070f0bf616f9","Type":"ContainerDied","Data":"e68e6fa84535abe342de25644126501492339073047f8b745287a4120aa3d513"}
Jul 15 16:57:04.802164 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:04.801929    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p2qjz" event={"ID":"00f13bdd-a991-489f-879a-019b72447eb5","Type":"ContainerStarted","Data":"aa056990fbf5805b53a814296d7cf0a4262833e0526f4e20234145910a1edd40"}
Jul 15 16:57:04.805386 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:04.805361    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mnvjv" event={"ID":"adca27dc-3730-43de-9940-070f0bf616f9","Type":"ContainerStarted","Data":"849990bca7536634d1975318baf8946e0fdc1113be8319b5f5424a93ae83f972"}
Jul 15 16:57:04.817283 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:04.817241    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-p2qjz" podStartSLOduration=1.558324847 podStartE2EDuration="5.817227356s" totalImagesPullingTime="4.258902509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:56:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:57:04.816318957 +0000 UTC m=+40.833732070" watchObservedRunningTime="2026-07-15 16:57:04.817227356 +0000 UTC m=+40.834640452"
Jul 15 16:57:04.836762 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:04.836715    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mnvjv" podStartSLOduration=8.414735788 podStartE2EDuration="39.836702554s" totalImagesPullingTime="31.421966766s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:57:04.835613675 +0000 UTC m=+40.853026771" watchObservedRunningTime="2026-07-15 16:57:04.836702554 +0000 UTC m=+40.854115650"
Jul 15 16:57:07.166291 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:07.166256    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls\") pod \"dns-default-f6q75\" (UID: \"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3\") " pod="openshift-dns/dns-default-f6q75"
Jul 15 16:57:07.166674 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:57:07.166395    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 16:57:07.166674 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:57:07.166458    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls podName:2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3 nodeName:}" failed. No retries permitted until 2026-07-15 16:57:15.166442601 +0000 UTC m=+51.183855675 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls") pod "dns-default-f6q75" (UID: "2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3") : secret "dns-default-metrics-tls" not found
Jul 15 16:57:15.212401 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:15.212365    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls\") pod \"dns-default-f6q75\" (UID: \"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3\") " pod="openshift-dns/dns-default-f6q75"
Jul 15 16:57:15.212885 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:57:15.212472    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 16:57:15.212885 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:57:15.212532    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls podName:2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3 nodeName:}" failed. No retries permitted until 2026-07-15 16:57:31.212517753 +0000 UTC m=+67.229930827 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls") pod "dns-default-f6q75" (UID: "2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3") : secret "dns-default-metrics-tls" not found
Jul 15 16:57:25.781475 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:25.781446    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kw2pl"
Jul 15 16:57:29.104180 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:29.104145    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret\") pod \"global-pull-secret-syncer-bnz8n\" (UID: \"ae13fea8-2c11-48da-b201-4007b9b6272c\") " pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:57:29.106545 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:29.106527    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 16:57:29.118979 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:29.118954    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae13fea8-2c11-48da-b201-4007b9b6272c-original-pull-secret\") pod \"global-pull-secret-syncer-bnz8n\" (UID: \"ae13fea8-2c11-48da-b201-4007b9b6272c\") " pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:57:29.205410 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:29.205368    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs\") pod \"network-metrics-daemon-7kv7g\" (UID: \"a62f9005-2c66-4d30-9102-121aae529be8\") " pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:57:29.207635 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:29.207618    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 16:57:29.216499 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:57:29.216481    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 15 16:57:29.216585 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:57:29.216533    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs podName:a62f9005-2c66-4d30-9102-121aae529be8 nodeName:}" failed. No retries permitted until 2026-07-15 16:58:33.216517973 +0000 UTC m=+129.233931046 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs") pod "network-metrics-daemon-7kv7g" (UID: "a62f9005-2c66-4d30-9102-121aae529be8") : secret "metrics-daemon-secret" not found
Jul 15 16:57:29.306083 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:29.306046    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tntpn\" (UniqueName: \"kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn\") pod \"network-check-target-9cncx\" (UID: \"8db7e638-ee27-4f89-a329-d6bb2fa60022\") " pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:57:29.309205 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:29.309187    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 16:57:29.319014 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:29.318996    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 16:57:29.329588 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:29.329553    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tntpn\" (UniqueName: \"kubernetes.io/projected/8db7e638-ee27-4f89-a329-d6bb2fa60022-kube-api-access-tntpn\") pod \"network-check-target-9cncx\" (UID: \"8db7e638-ee27-4f89-a329-d6bb2fa60022\") " pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:57:29.332390 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:29.332361    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bnz8n"
Jul 15 16:57:29.342575 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:29.342544    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7h7tx\""
Jul 15 16:57:29.350519 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:29.350503    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:57:29.463219 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:29.463168    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bnz8n"]
Jul 15 16:57:29.466900 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:57:29.466872    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae13fea8_2c11_48da_b201_4007b9b6272c.slice/crio-752c66d93a78dd4a12092291e8e6b72776c488cec35162c9f00bce8d65039d7d WatchSource:0}: Error finding container 752c66d93a78dd4a12092291e8e6b72776c488cec35162c9f00bce8d65039d7d: Status 404 returned error can't find the container with id 752c66d93a78dd4a12092291e8e6b72776c488cec35162c9f00bce8d65039d7d
Jul 15 16:57:29.479402 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:29.479380    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9cncx"]
Jul 15 16:57:29.482311 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:57:29.482288    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8db7e638_ee27_4f89_a329_d6bb2fa60022.slice/crio-861c868b8ed8c59d3a3ecf4ebf4bbc8e5a0a1c0fa2e5d4fc8bb28ec3cde1501f WatchSource:0}: Error finding container 861c868b8ed8c59d3a3ecf4ebf4bbc8e5a0a1c0fa2e5d4fc8bb28ec3cde1501f: Status 404 returned error can't find the container with id 861c868b8ed8c59d3a3ecf4ebf4bbc8e5a0a1c0fa2e5d4fc8bb28ec3cde1501f
Jul 15 16:57:29.864100 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:29.864067    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9cncx" event={"ID":"8db7e638-ee27-4f89-a329-d6bb2fa60022","Type":"ContainerStarted","Data":"861c868b8ed8c59d3a3ecf4ebf4bbc8e5a0a1c0fa2e5d4fc8bb28ec3cde1501f"}
Jul 15 16:57:29.865152 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:29.865130    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bnz8n" event={"ID":"ae13fea8-2c11-48da-b201-4007b9b6272c","Type":"ContainerStarted","Data":"752c66d93a78dd4a12092291e8e6b72776c488cec35162c9f00bce8d65039d7d"}
Jul 15 16:57:31.218871 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:31.218782    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls\") pod \"dns-default-f6q75\" (UID: \"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3\") " pod="openshift-dns/dns-default-f6q75"
Jul 15 16:57:31.219368 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:57:31.218957    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 16:57:31.219368 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:57:31.219042    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls podName:2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3 nodeName:}" failed. No retries permitted until 2026-07-15 16:58:03.219020804 +0000 UTC m=+99.236433881 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls") pod "dns-default-f6q75" (UID: "2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3") : secret "dns-default-metrics-tls" not found
Jul 15 16:57:34.879298 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:34.879261    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9cncx" event={"ID":"8db7e638-ee27-4f89-a329-d6bb2fa60022","Type":"ContainerStarted","Data":"a8026e71cdf1f9de2f7ab316f7d3c326dd8206d1ee324d81d91c92acbba0f9ff"}
Jul 15 16:57:34.879756 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:34.879324    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:57:34.880807 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:34.880776    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bnz8n" event={"ID":"ae13fea8-2c11-48da-b201-4007b9b6272c","Type":"ContainerStarted","Data":"0cc5980a57820c621604f85bacda250be5357553aac9ceb5a3fca8eb902dd196"}
Jul 15 16:57:34.894073 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:34.894033    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9cncx" podStartSLOduration=65.247129059 podStartE2EDuration="1m9.89401827s" totalImagesPullingTime="4.646889211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:57:34.893223859 +0000 UTC m=+70.910636974" watchObservedRunningTime="2026-07-15 16:57:34.89401827 +0000 UTC m=+70.911431343"
Jul 15 16:57:34.906885 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:34.906844    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bnz8n" podStartSLOduration=65.233020225 podStartE2EDuration="1m9.906828433s" totalImagesPullingTime="4.673808208s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:57:34.906600211 +0000 UTC m=+70.924013325" watchObservedRunningTime="2026-07-15 16:57:34.906828433 +0000 UTC m=+70.924241529"
Jul 15 16:57:36.180913 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.180861    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b"]
Jul 15 16:57:36.182892 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.182876    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"]
Jul 15 16:57:36.183006 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.182990    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b"
Jul 15 16:57:36.184737 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.184722    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"
Jul 15 16:57:36.185332 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.185313    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 15 16:57:36.185597 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.185580    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 15 16:57:36.185686 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.185583    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 15 16:57:36.186777 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.186759    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 15 16:57:36.187164 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.187148    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 15 16:57:36.187293 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.187279    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 15 16:57:36.187357 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.187282    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 15 16:57:36.187420 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.187365    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 15 16:57:36.192952 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.192930    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b"]
Jul 15 16:57:36.195845 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.195826    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"]
Jul 15 16:57:36.256399 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.256371    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bc29048c-f17e-40c8-bd52-5ea567cb9225-klusterlet-config\") pod \"klusterlet-addon-workmgr-74d87b7d48-8r74b\" (UID: \"bc29048c-f17e-40c8-bd52-5ea567cb9225\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b"
Jul 15 16:57:36.256533 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.256404    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5pjf\" (UniqueName: \"kubernetes.io/projected/bc29048c-f17e-40c8-bd52-5ea567cb9225-kube-api-access-m5pjf\") pod \"klusterlet-addon-workmgr-74d87b7d48-8r74b\" (UID: \"bc29048c-f17e-40c8-bd52-5ea567cb9225\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b"
Jul 15 16:57:36.256533 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.256428    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89468\" (UniqueName: \"kubernetes.io/projected/9ce368e3-4392-458d-aeee-c8e06c545966-kube-api-access-89468\") pod \"cluster-proxy-proxy-agent-5449f8fcb7-vqg9s\" (UID: \"9ce368e3-4392-458d-aeee-c8e06c545966\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"
Jul 15 16:57:36.256533 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.256511    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9ce368e3-4392-458d-aeee-c8e06c545966-hub\") pod \"cluster-proxy-proxy-agent-5449f8fcb7-vqg9s\" (UID: \"9ce368e3-4392-458d-aeee-c8e06c545966\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"
Jul 15 16:57:36.256533 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.256529    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc29048c-f17e-40c8-bd52-5ea567cb9225-tmp\") pod \"klusterlet-addon-workmgr-74d87b7d48-8r74b\" (UID: \"bc29048c-f17e-40c8-bd52-5ea567cb9225\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b"
Jul 15 16:57:36.256688 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.256574    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9ce368e3-4392-458d-aeee-c8e06c545966-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5449f8fcb7-vqg9s\" (UID: \"9ce368e3-4392-458d-aeee-c8e06c545966\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"
Jul 15 16:57:36.256688 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.256633    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce368e3-4392-458d-aeee-c8e06c545966-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5449f8fcb7-vqg9s\" (UID: \"9ce368e3-4392-458d-aeee-c8e06c545966\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"
Jul 15 16:57:36.256688 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.256674    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9ce368e3-4392-458d-aeee-c8e06c545966-ca\") pod \"cluster-proxy-proxy-agent-5449f8fcb7-vqg9s\" (UID: \"9ce368e3-4392-458d-aeee-c8e06c545966\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"
Jul 15 16:57:36.256776 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.256693    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9ce368e3-4392-458d-aeee-c8e06c545966-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5449f8fcb7-vqg9s\" (UID: \"9ce368e3-4392-458d-aeee-c8e06c545966\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"
Jul 15 16:57:36.357615 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.357589    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce368e3-4392-458d-aeee-c8e06c545966-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5449f8fcb7-vqg9s\" (UID: \"9ce368e3-4392-458d-aeee-c8e06c545966\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"
Jul 15 16:57:36.357738 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.357618    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9ce368e3-4392-458d-aeee-c8e06c545966-ca\") pod \"cluster-proxy-proxy-agent-5449f8fcb7-vqg9s\" (UID: \"9ce368e3-4392-458d-aeee-c8e06c545966\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"
Jul 15 16:57:36.357738 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.357637    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9ce368e3-4392-458d-aeee-c8e06c545966-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5449f8fcb7-vqg9s\" (UID: \"9ce368e3-4392-458d-aeee-c8e06c545966\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"
Jul 15 16:57:36.357738 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.357655    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bc29048c-f17e-40c8-bd52-5ea567cb9225-klusterlet-config\") pod \"klusterlet-addon-workmgr-74d87b7d48-8r74b\" (UID: \"bc29048c-f17e-40c8-bd52-5ea567cb9225\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b"
Jul 15 16:57:36.357867 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.357780    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5pjf\" (UniqueName: \"kubernetes.io/projected/bc29048c-f17e-40c8-bd52-5ea567cb9225-kube-api-access-m5pjf\") pod \"klusterlet-addon-workmgr-74d87b7d48-8r74b\" (UID: \"bc29048c-f17e-40c8-bd52-5ea567cb9225\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b"
Jul 15 16:57:36.357867 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.357814    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89468\" (UniqueName: \"kubernetes.io/projected/9ce368e3-4392-458d-aeee-c8e06c545966-kube-api-access-89468\") pod \"cluster-proxy-proxy-agent-5449f8fcb7-vqg9s\" (UID: \"9ce368e3-4392-458d-aeee-c8e06c545966\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"
Jul 15 16:57:36.357867 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.357860    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9ce368e3-4392-458d-aeee-c8e06c545966-hub\") pod \"cluster-proxy-proxy-agent-5449f8fcb7-vqg9s\" (UID: \"9ce368e3-4392-458d-aeee-c8e06c545966\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"
Jul 15 16:57:36.358004 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.357888    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc29048c-f17e-40c8-bd52-5ea567cb9225-tmp\") pod \"klusterlet-addon-workmgr-74d87b7d48-8r74b\" (UID: \"bc29048c-f17e-40c8-bd52-5ea567cb9225\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b"
Jul 15 16:57:36.358004 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.357933    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9ce368e3-4392-458d-aeee-c8e06c545966-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5449f8fcb7-vqg9s\" (UID: \"9ce368e3-4392-458d-aeee-c8e06c545966\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"
Jul 15 16:57:36.358398 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.358371    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc29048c-f17e-40c8-bd52-5ea567cb9225-tmp\") pod \"klusterlet-addon-workmgr-74d87b7d48-8r74b\" (UID: \"bc29048c-f17e-40c8-bd52-5ea567cb9225\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b"
Jul 15 16:57:36.358511 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.358460    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce368e3-4392-458d-aeee-c8e06c545966-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5449f8fcb7-vqg9s\" (UID: \"9ce368e3-4392-458d-aeee-c8e06c545966\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"
Jul 15 16:57:36.360093 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.360073    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9ce368e3-4392-458d-aeee-c8e06c545966-ca\") pod \"cluster-proxy-proxy-agent-5449f8fcb7-vqg9s\" (UID: \"9ce368e3-4392-458d-aeee-c8e06c545966\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"
Jul 15 16:57:36.360678 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.360654    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9ce368e3-4392-458d-aeee-c8e06c545966-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5449f8fcb7-vqg9s\" (UID: \"9ce368e3-4392-458d-aeee-c8e06c545966\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"
Jul 15 16:57:36.360678 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.360672    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bc29048c-f17e-40c8-bd52-5ea567cb9225-klusterlet-config\") pod \"klusterlet-addon-workmgr-74d87b7d48-8r74b\" (UID: \"bc29048c-f17e-40c8-bd52-5ea567cb9225\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b"
Jul 15 16:57:36.360799 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.360712    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9ce368e3-4392-458d-aeee-c8e06c545966-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5449f8fcb7-vqg9s\" (UID: \"9ce368e3-4392-458d-aeee-c8e06c545966\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"
Jul 15 16:57:36.360799 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.360760    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9ce368e3-4392-458d-aeee-c8e06c545966-hub\") pod \"cluster-proxy-proxy-agent-5449f8fcb7-vqg9s\" (UID: \"9ce368e3-4392-458d-aeee-c8e06c545966\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"
Jul 15 16:57:36.367125 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.367102    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89468\" (UniqueName: \"kubernetes.io/projected/9ce368e3-4392-458d-aeee-c8e06c545966-kube-api-access-89468\") pod \"cluster-proxy-proxy-agent-5449f8fcb7-vqg9s\" (UID: \"9ce368e3-4392-458d-aeee-c8e06c545966\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"
Jul 15 16:57:36.367407 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.367388    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5pjf\" (UniqueName: \"kubernetes.io/projected/bc29048c-f17e-40c8-bd52-5ea567cb9225-kube-api-access-m5pjf\") pod \"klusterlet-addon-workmgr-74d87b7d48-8r74b\" (UID: \"bc29048c-f17e-40c8-bd52-5ea567cb9225\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b"
Jul 15 16:57:36.493798 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.493722    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b"
Jul 15 16:57:36.519897 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.519875    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"
Jul 15 16:57:36.625292 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.625268    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b"]
Jul 15 16:57:36.628231 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:57:36.628208    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc29048c_f17e_40c8_bd52_5ea567cb9225.slice/crio-34bdf0a2290259da3c1c914a9c1698f297ccc2ad8e2d2827adc64c0ada480194 WatchSource:0}: Error finding container 34bdf0a2290259da3c1c914a9c1698f297ccc2ad8e2d2827adc64c0ada480194: Status 404 returned error can't find the container with id 34bdf0a2290259da3c1c914a9c1698f297ccc2ad8e2d2827adc64c0ada480194
Jul 15 16:57:36.644482 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.644461    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"]
Jul 15 16:57:36.647200 ip-10-0-142-51 kubenswrapper[2575]: W0715 16:57:36.647179    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ce368e3_4392_458d_aeee_c8e06c545966.slice/crio-2292e1f475360730c0849722552483399cb485fd827f555f2f1eb44b3da68f6f WatchSource:0}: Error finding container 2292e1f475360730c0849722552483399cb485fd827f555f2f1eb44b3da68f6f: Status 404 returned error can't find the container with id 2292e1f475360730c0849722552483399cb485fd827f555f2f1eb44b3da68f6f
Jul 15 16:57:36.886828 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.886800    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b" event={"ID":"bc29048c-f17e-40c8-bd52-5ea567cb9225","Type":"ContainerStarted","Data":"34bdf0a2290259da3c1c914a9c1698f297ccc2ad8e2d2827adc64c0ada480194"}
Jul 15 16:57:36.887817 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:36.887796    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s" event={"ID":"9ce368e3-4392-458d-aeee-c8e06c545966","Type":"ContainerStarted","Data":"2292e1f475360730c0849722552483399cb485fd827f555f2f1eb44b3da68f6f"}
Jul 15 16:57:41.905591 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:41.905490    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s" event={"ID":"9ce368e3-4392-458d-aeee-c8e06c545966","Type":"ContainerStarted","Data":"6098d937f33442ed099e937eedfa8b6eae403c573c293375a94ba8608d811639"}
Jul 15 16:57:41.905591 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:41.905549    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s" event={"ID":"9ce368e3-4392-458d-aeee-c8e06c545966","Type":"ContainerStarted","Data":"86fd59adaa95624553882a21c459066b3ff6478e1e9077d2c5270970c5e3a53a"}
Jul 15 16:57:41.905591 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:41.905588    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s" event={"ID":"9ce368e3-4392-458d-aeee-c8e06c545966","Type":"ContainerStarted","Data":"b4f65874e6cc703ff8576122578ae4e5c5643e97a854669a54a4898591709850"}
Jul 15 16:57:41.906922 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:41.906898    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b" event={"ID":"bc29048c-f17e-40c8-bd52-5ea567cb9225","Type":"ContainerStarted","Data":"7066c61aa6f91c3eb606319b00c602ec370a8b351819b65eba3e4c07d9f5da14"}
Jul 15 16:57:41.907113 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:41.907095    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b"
Jul 15 16:57:41.908490 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:41.908472    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b"
Jul 15 16:57:41.924240 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:41.924200    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s" podStartSLOduration=0.941213164 podStartE2EDuration="5.924188215s" totalImagesPullingTime="4.982975051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:57:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:57:41.922658521 +0000 UTC m=+77.940071616" watchObservedRunningTime="2026-07-15 16:57:41.924188215 +0000 UTC m=+77.941601311"
Jul 15 16:57:41.937738 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:57:41.937702    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b" podStartSLOduration=0.934200201 podStartE2EDuration="5.937686086s" totalImagesPullingTime="5.003485885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:57:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:57:41.937199028 +0000 UTC m=+77.954612125" watchObservedRunningTime="2026-07-15 16:57:41.937686086 +0000 UTC m=+77.955099185"
Jul 15 16:58:03.244276 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:58:03.244153    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls\") pod \"dns-default-f6q75\" (UID: \"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3\") " pod="openshift-dns/dns-default-f6q75"
Jul 15 16:58:03.244276 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:58:03.244272    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 16:58:03.244819 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:58:03.244357    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls podName:2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:07.244343936 +0000 UTC m=+163.261757010 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls") pod "dns-default-f6q75" (UID: "2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3") : secret "dns-default-metrics-tls" not found
Jul 15 16:58:05.886271 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:58:05.886241    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9cncx"
Jul 15 16:58:33.242086 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:58:33.242049    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs\") pod \"network-metrics-daemon-7kv7g\" (UID: \"a62f9005-2c66-4d30-9102-121aae529be8\") " pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:58:33.242530 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:58:33.242177    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 15 16:58:33.242530 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:58:33.242229    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs podName:a62f9005-2c66-4d30-9102-121aae529be8 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:35.242214264 +0000 UTC m=+251.259627338 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs") pod "network-metrics-daemon-7kv7g" (UID: "a62f9005-2c66-4d30-9102-121aae529be8") : secret "metrics-daemon-secret" not found
Jul 15 16:59:02.388309 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:59:02.388271    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-f6q75" podUID="2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3"
Jul 15 16:59:03.150381 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:59:03.150352    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f6q75"
Jul 15 16:59:03.544795 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:59:03.544706    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-7kv7g" podUID="a62f9005-2c66-4d30-9102-121aae529be8"
Jul 15 16:59:07.278663 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:59:07.278623    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls\") pod \"dns-default-f6q75\" (UID: \"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3\") " pod="openshift-dns/dns-default-f6q75"
Jul 15 16:59:07.279050 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:59:07.278770    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 16:59:07.279050 ip-10-0-142-51 kubenswrapper[2575]: E0715 16:59:07.278839    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls podName:2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3 nodeName:}" failed. No retries permitted until 2026-07-15 17:01:09.278823675 +0000 UTC m=+285.296236748 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls") pod "dns-default-f6q75" (UID: "2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3") : secret "dns-default-metrics-tls" not found
Jul 15 16:59:17.519651 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:59:17.519578    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 16:59:41.908376 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:59:41.908327    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b" podUID="bc29048c-f17e-40c8-bd52-5ea567cb9225" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.8:8000/readyz\": dial tcp 10.134.0.8:8000: connect: connection refused"
Jul 15 16:59:42.266656 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:59:42.266624    2575 generic.go:356] "Generic (PLEG): container finished" podID="bc29048c-f17e-40c8-bd52-5ea567cb9225" containerID="7066c61aa6f91c3eb606319b00c602ec370a8b351819b65eba3e4c07d9f5da14" exitCode=1
Jul 15 16:59:42.266806 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:59:42.266695    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b" event={"ID":"bc29048c-f17e-40c8-bd52-5ea567cb9225","Type":"ContainerDied","Data":"7066c61aa6f91c3eb606319b00c602ec370a8b351819b65eba3e4c07d9f5da14"}
Jul 15 16:59:42.267032 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:59:42.267018    2575 scope.go:117] "RemoveContainer" containerID="7066c61aa6f91c3eb606319b00c602ec370a8b351819b65eba3e4c07d9f5da14"
Jul 15 16:59:43.271062 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:59:43.271030    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b" event={"ID":"bc29048c-f17e-40c8-bd52-5ea567cb9225","Type":"ContainerStarted","Data":"809876bbca0988bed37228d2513fdb17044fcd1163d2199bbc2c2269ff6d6a84"}
Jul 15 16:59:43.271447 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:59:43.271309    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b"
Jul 15 16:59:43.271908 ip-10-0-142-51 kubenswrapper[2575]: I0715 16:59:43.271894    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d87b7d48-8r74b"
Jul 15 17:00:35.268663 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:35.268625    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs\") pod \"network-metrics-daemon-7kv7g\" (UID: \"a62f9005-2c66-4d30-9102-121aae529be8\") " pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 17:00:35.269142 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:35.268774    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 15 17:00:35.269142 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:35.268843    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs podName:a62f9005-2c66-4d30-9102-121aae529be8 nodeName:}" failed. No retries permitted until 2026-07-15 17:02:37.268826164 +0000 UTC m=+373.286239238 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs") pod "network-metrics-daemon-7kv7g" (UID: "a62f9005-2c66-4d30-9102-121aae529be8") : secret "metrics-daemon-secret" not found
Jul 15 17:00:43.014701 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.014667    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tlgvp"]
Jul 15 17:00:43.016532 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.016515    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:43.017085 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.017058    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fee99ffe-3358-4efa-8014-a66f6a973724-data-volume\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:43.017202 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.017089    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lbf6\" (UniqueName: \"kubernetes.io/projected/fee99ffe-3358-4efa-8014-a66f6a973724-kube-api-access-6lbf6\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:43.017202 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.017116    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fee99ffe-3358-4efa-8014-a66f6a973724-crio-socket\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:43.017202 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.017139    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fee99ffe-3358-4efa-8014-a66f6a973724-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:43.017348 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.017216    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fee99ffe-3358-4efa-8014-a66f6a973724-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:43.020078 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.020055    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 15 17:00:43.020078 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.020070    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tmfrj\""
Jul 15 17:00:43.020219 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.020055    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 15 17:00:43.020219 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.020057    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 15 17:00:43.021050 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.021034    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 15 17:00:43.034035 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.034015    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tlgvp"]
Jul 15 17:00:43.117501 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.117467    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fee99ffe-3358-4efa-8014-a66f6a973724-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:43.117656 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.117531    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fee99ffe-3358-4efa-8014-a66f6a973724-data-volume\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:43.117656 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.117552    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lbf6\" (UniqueName: \"kubernetes.io/projected/fee99ffe-3358-4efa-8014-a66f6a973724-kube-api-access-6lbf6\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:43.117656 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.117590    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fee99ffe-3358-4efa-8014-a66f6a973724-crio-socket\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:43.117656 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:43.117593    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 17:00:43.117656 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.117648    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fee99ffe-3358-4efa-8014-a66f6a973724-crio-socket\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:43.117889 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.117670    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fee99ffe-3358-4efa-8014-a66f6a973724-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:43.117889 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:43.117678    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fee99ffe-3358-4efa-8014-a66f6a973724-kube-rbac-proxy-cm podName:fee99ffe-3358-4efa-8014-a66f6a973724 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:43.617654985 +0000 UTC m=+259.635068058 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fee99ffe-3358-4efa-8014-a66f6a973724-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tlgvp" (UID: "fee99ffe-3358-4efa-8014-a66f6a973724") : configmap "kube-rbac-proxy" not found
Jul 15 17:00:43.117889 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:43.117747    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 17:00:43.117889 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:43.117790    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fee99ffe-3358-4efa-8014-a66f6a973724-insights-runtime-extractor-tls podName:fee99ffe-3358-4efa-8014-a66f6a973724 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:43.617778745 +0000 UTC m=+259.635191819 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fee99ffe-3358-4efa-8014-a66f6a973724-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tlgvp" (UID: "fee99ffe-3358-4efa-8014-a66f6a973724") : secret "insights-runtime-extractor-tls" not found
Jul 15 17:00:43.117889 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.117873    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fee99ffe-3358-4efa-8014-a66f6a973724-data-volume\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:43.126196 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.126177    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lbf6\" (UniqueName: \"kubernetes.io/projected/fee99ffe-3358-4efa-8014-a66f6a973724-kube-api-access-6lbf6\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:43.621981 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.621950    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fee99ffe-3358-4efa-8014-a66f6a973724-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:43.622141 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:43.621991    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fee99ffe-3358-4efa-8014-a66f6a973724-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:43.622141 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:43.622057    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 17:00:43.622141 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:43.622075    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 17:00:43.622141 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:43.622098    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fee99ffe-3358-4efa-8014-a66f6a973724-kube-rbac-proxy-cm podName:fee99ffe-3358-4efa-8014-a66f6a973724 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:44.622085038 +0000 UTC m=+260.639498112 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fee99ffe-3358-4efa-8014-a66f6a973724-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tlgvp" (UID: "fee99ffe-3358-4efa-8014-a66f6a973724") : configmap "kube-rbac-proxy" not found
Jul 15 17:00:43.622141 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:43.622128    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fee99ffe-3358-4efa-8014-a66f6a973724-insights-runtime-extractor-tls podName:fee99ffe-3358-4efa-8014-a66f6a973724 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:44.62211579 +0000 UTC m=+260.639528864 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fee99ffe-3358-4efa-8014-a66f6a973724-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tlgvp" (UID: "fee99ffe-3358-4efa-8014-a66f6a973724") : secret "insights-runtime-extractor-tls" not found
Jul 15 17:00:44.631213 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:44.631180    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fee99ffe-3358-4efa-8014-a66f6a973724-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:44.631599 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:44.631221    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fee99ffe-3358-4efa-8014-a66f6a973724-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:44.631599 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:44.631306    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 17:00:44.631599 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:44.631319    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 17:00:44.631599 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:44.631354    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fee99ffe-3358-4efa-8014-a66f6a973724-kube-rbac-proxy-cm podName:fee99ffe-3358-4efa-8014-a66f6a973724 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:46.631341155 +0000 UTC m=+262.648754229 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fee99ffe-3358-4efa-8014-a66f6a973724-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tlgvp" (UID: "fee99ffe-3358-4efa-8014-a66f6a973724") : configmap "kube-rbac-proxy" not found
Jul 15 17:00:44.631599 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:44.631377    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fee99ffe-3358-4efa-8014-a66f6a973724-insights-runtime-extractor-tls podName:fee99ffe-3358-4efa-8014-a66f6a973724 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:46.631364605 +0000 UTC m=+262.648777679 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fee99ffe-3358-4efa-8014-a66f6a973724-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tlgvp" (UID: "fee99ffe-3358-4efa-8014-a66f6a973724") : secret "insights-runtime-extractor-tls" not found
Jul 15 17:00:45.771872 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:45.771847    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jdrr4_c1de2bfc-4272-4761-b27a-76d5c4f9949e/dns-node-resolver/0.log"
Jul 15 17:00:46.572771 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:46.572739    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2brrq_5d97b3f4-a52b-4bd8-bf1a-ab72f5304ea4/node-ca/0.log"
Jul 15 17:00:46.646481 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:46.646439    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fee99ffe-3358-4efa-8014-a66f6a973724-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:46.646639 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:46.646501    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fee99ffe-3358-4efa-8014-a66f6a973724-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:46.646639 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:46.646592    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 17:00:46.646754 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:46.646646    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fee99ffe-3358-4efa-8014-a66f6a973724-insights-runtime-extractor-tls podName:fee99ffe-3358-4efa-8014-a66f6a973724 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:50.646630091 +0000 UTC m=+266.664043166 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fee99ffe-3358-4efa-8014-a66f6a973724-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tlgvp" (UID: "fee99ffe-3358-4efa-8014-a66f6a973724") : secret "insights-runtime-extractor-tls" not found
Jul 15 17:00:46.646754 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:46.646594    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 17:00:46.646754 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:46.646745    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fee99ffe-3358-4efa-8014-a66f6a973724-kube-rbac-proxy-cm podName:fee99ffe-3358-4efa-8014-a66f6a973724 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:50.646726867 +0000 UTC m=+266.664139948 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fee99ffe-3358-4efa-8014-a66f6a973724-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tlgvp" (UID: "fee99ffe-3358-4efa-8014-a66f6a973724") : configmap "kube-rbac-proxy" not found
Jul 15 17:00:47.572379 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:47.572346    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p2qjz_00f13bdd-a991-489f-879a-019b72447eb5/serve-healthcheck-canary/0.log"
Jul 15 17:00:50.673648 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:50.673614    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fee99ffe-3358-4efa-8014-a66f6a973724-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:50.674025 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:50.673659    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fee99ffe-3358-4efa-8014-a66f6a973724-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:50.674025 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:50.673760    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 17:00:50.674025 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:50.673764    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 17:00:50.674025 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:50.673821    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fee99ffe-3358-4efa-8014-a66f6a973724-insights-runtime-extractor-tls podName:fee99ffe-3358-4efa-8014-a66f6a973724 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:58.673806311 +0000 UTC m=+274.691219385 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fee99ffe-3358-4efa-8014-a66f6a973724-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tlgvp" (UID: "fee99ffe-3358-4efa-8014-a66f6a973724") : secret "insights-runtime-extractor-tls" not found
Jul 15 17:00:50.674025 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:50.673835    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fee99ffe-3358-4efa-8014-a66f6a973724-kube-rbac-proxy-cm podName:fee99ffe-3358-4efa-8014-a66f6a973724 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:58.673829153 +0000 UTC m=+274.691242228 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fee99ffe-3358-4efa-8014-a66f6a973724-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tlgvp" (UID: "fee99ffe-3358-4efa-8014-a66f6a973724") : configmap "kube-rbac-proxy" not found
Jul 15 17:00:58.728550 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:58.728514    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fee99ffe-3358-4efa-8014-a66f6a973724-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:58.728950 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:58.728618    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fee99ffe-3358-4efa-8014-a66f6a973724-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:00:58.728950 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:58.728712    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 17:00:58.728950 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:00:58.728778    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fee99ffe-3358-4efa-8014-a66f6a973724-insights-runtime-extractor-tls podName:fee99ffe-3358-4efa-8014-a66f6a973724 nodeName:}" failed. No retries permitted until 2026-07-15 17:01:14.728765009 +0000 UTC m=+290.746178082 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fee99ffe-3358-4efa-8014-a66f6a973724-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tlgvp" (UID: "fee99ffe-3358-4efa-8014-a66f6a973724") : secret "insights-runtime-extractor-tls" not found
Jul 15 17:00:58.729058 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:00:58.729024    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fee99ffe-3358-4efa-8014-a66f6a973724-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:01:06.151446 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:01:06.151391    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-f6q75" podUID="2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3"
Jul 15 17:01:06.499604 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:06.499515    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f6q75"
Jul 15 17:01:09.300827 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:09.300791    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls\") pod \"dns-default-f6q75\" (UID: \"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3\") " pod="openshift-dns/dns-default-f6q75"
Jul 15 17:01:09.303039 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:09.303013    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3-metrics-tls\") pod \"dns-default-f6q75\" (UID: \"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3\") " pod="openshift-dns/dns-default-f6q75"
Jul 15 17:01:09.502478 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:09.502452    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lllhc\""
Jul 15 17:01:09.510955 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:09.510929    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f6q75"
Jul 15 17:01:09.627760 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:09.627732    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f6q75"]
Jul 15 17:01:09.631200 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:01:09.631173    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cd5fea3_92ab_4a77_8b89_a91ef90aa2a3.slice/crio-76d85adc95e9b5768e6d018b6cdffb5701cc5acd13462ff771e465ed3d4fb61d WatchSource:0}: Error finding container 76d85adc95e9b5768e6d018b6cdffb5701cc5acd13462ff771e465ed3d4fb61d: Status 404 returned error can't find the container with id 76d85adc95e9b5768e6d018b6cdffb5701cc5acd13462ff771e465ed3d4fb61d
Jul 15 17:01:10.511355 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:10.511319    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f6q75" event={"ID":"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3","Type":"ContainerStarted","Data":"76d85adc95e9b5768e6d018b6cdffb5701cc5acd13462ff771e465ed3d4fb61d"}
Jul 15 17:01:11.516134 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:11.516102    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f6q75" event={"ID":"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3","Type":"ContainerStarted","Data":"1b409fc29b7e55babc74a649f99388ab949c67efe9413e7dc1ee66718facbdca"}
Jul 15 17:01:11.516502 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:11.516153    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f6q75" event={"ID":"2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3","Type":"ContainerStarted","Data":"d65b98f282a367bdd56dfb395978720eff99c6b606129915790feae1b15ac106"}
Jul 15 17:01:11.516502 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:11.516243    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-f6q75"
Jul 15 17:01:11.532891 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:11.532848    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-f6q75" podStartSLOduration=251.100314963 podStartE2EDuration="4m12.532834398s" totalImagesPullingTime="1.432519435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:56:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:11.531477949 +0000 UTC m=+287.548891046" watchObservedRunningTime="2026-07-15 17:01:11.532834398 +0000 UTC m=+287.550247499"
Jul 15 17:01:14.737326 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:14.737272    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fee99ffe-3358-4efa-8014-a66f6a973724-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:01:14.739549 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:14.739531    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fee99ffe-3358-4efa-8014-a66f6a973724-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tlgvp\" (UID: \"fee99ffe-3358-4efa-8014-a66f6a973724\") " pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:01:14.829749 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:14.829709    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tlgvp"
Jul 15 17:01:14.952285 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:14.952256    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tlgvp"]
Jul 15 17:01:14.955621 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:01:14.955599    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfee99ffe_3358_4efa_8014_a66f6a973724.slice/crio-53a50dccf4f074856c670634e6f5ced0d31434278758b426fe939e385f3fbad4 WatchSource:0}: Error finding container 53a50dccf4f074856c670634e6f5ced0d31434278758b426fe939e385f3fbad4: Status 404 returned error can't find the container with id 53a50dccf4f074856c670634e6f5ced0d31434278758b426fe939e385f3fbad4
Jul 15 17:01:15.162365 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.160924    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58845c4f86-llrx9"]
Jul 15 17:01:15.164422 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.164396    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.166700 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.166659    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 15 17:01:15.166792 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.166694    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 15 17:01:15.166792 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.166723    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 15 17:01:15.167098 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.167080    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-d5smq\""
Jul 15 17:01:15.167199 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.167082    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 15 17:01:15.167813 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.167789    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 15 17:01:15.167913 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.167814    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 15 17:01:15.168039 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.168023    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 15 17:01:15.172268 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.172203    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 15 17:01:15.173812 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.173789    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58845c4f86-llrx9"]
Jul 15 17:01:15.242177 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.242136    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-oauth-serving-cert\") pod \"console-58845c4f86-llrx9\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") " pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.242177 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.242182    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2c59b417-2f6d-4949-ac54-94e9889f8379-console-serving-cert\") pod \"console-58845c4f86-llrx9\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") " pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.242404 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.242231    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-console-config\") pod \"console-58845c4f86-llrx9\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") " pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.242404 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.242257    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-service-ca\") pod \"console-58845c4f86-llrx9\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") " pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.242404 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.242342    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2c59b417-2f6d-4949-ac54-94e9889f8379-console-oauth-config\") pod \"console-58845c4f86-llrx9\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") " pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.242404 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.242377    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-trusted-ca-bundle\") pod \"console-58845c4f86-llrx9\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") " pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.242404 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.242397    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp2nq\" (UniqueName: \"kubernetes.io/projected/2c59b417-2f6d-4949-ac54-94e9889f8379-kube-api-access-gp2nq\") pod \"console-58845c4f86-llrx9\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") " pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.343490 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.343454    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-oauth-serving-cert\") pod \"console-58845c4f86-llrx9\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") " pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.343663 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.343494    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2c59b417-2f6d-4949-ac54-94e9889f8379-console-serving-cert\") pod \"console-58845c4f86-llrx9\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") " pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.343663 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.343525    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-console-config\") pod \"console-58845c4f86-llrx9\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") " pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.343663 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.343542    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-service-ca\") pod \"console-58845c4f86-llrx9\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") " pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.343663 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.343599    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2c59b417-2f6d-4949-ac54-94e9889f8379-console-oauth-config\") pod \"console-58845c4f86-llrx9\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") " pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.343663 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.343617    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-trusted-ca-bundle\") pod \"console-58845c4f86-llrx9\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") " pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.343663 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.343660    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp2nq\" (UniqueName: \"kubernetes.io/projected/2c59b417-2f6d-4949-ac54-94e9889f8379-kube-api-access-gp2nq\") pod \"console-58845c4f86-llrx9\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") " pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.344356 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.344322    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-oauth-serving-cert\") pod \"console-58845c4f86-llrx9\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") " pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.344492 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.344393    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-console-config\") pod \"console-58845c4f86-llrx9\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") " pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.344492 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.344404    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-service-ca\") pod \"console-58845c4f86-llrx9\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") " pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.344764 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.344745    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-trusted-ca-bundle\") pod \"console-58845c4f86-llrx9\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") " pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.346053 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.346022    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2c59b417-2f6d-4949-ac54-94e9889f8379-console-serving-cert\") pod \"console-58845c4f86-llrx9\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") " pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.346183 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.346146    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2c59b417-2f6d-4949-ac54-94e9889f8379-console-oauth-config\") pod \"console-58845c4f86-llrx9\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") " pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.354423 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.354404    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp2nq\" (UniqueName: \"kubernetes.io/projected/2c59b417-2f6d-4949-ac54-94e9889f8379-kube-api-access-gp2nq\") pod \"console-58845c4f86-llrx9\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") " pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.364037 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.364016    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6w956"]
Jul 15 17:01:15.366708 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.366693    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6w956"
Jul 15 17:01:15.369471 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.369451    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 15 17:01:15.369583 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.369485    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-2fpbh\""
Jul 15 17:01:15.375365 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.375344    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6w956"]
Jul 15 17:01:15.444513 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.444434    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e3efe260-102a-456d-9c7d-a47277cf1f35-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-6w956\" (UID: \"e3efe260-102a-456d-9c7d-a47277cf1f35\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6w956"
Jul 15 17:01:15.474474 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.474444    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:15.530244 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.530215    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tlgvp" event={"ID":"fee99ffe-3358-4efa-8014-a66f6a973724","Type":"ContainerStarted","Data":"53a50dccf4f074856c670634e6f5ced0d31434278758b426fe939e385f3fbad4"}
Jul 15 17:01:15.545573 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.545531    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e3efe260-102a-456d-9c7d-a47277cf1f35-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-6w956\" (UID: \"e3efe260-102a-456d-9c7d-a47277cf1f35\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6w956"
Jul 15 17:01:15.547860 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.547753    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e3efe260-102a-456d-9c7d-a47277cf1f35-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-6w956\" (UID: \"e3efe260-102a-456d-9c7d-a47277cf1f35\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6w956"
Jul 15 17:01:15.594262 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.594234    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58845c4f86-llrx9"]
Jul 15 17:01:15.596691 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:01:15.596670    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c59b417_2f6d_4949_ac54_94e9889f8379.slice/crio-88ad2ed36123d8d79effe85181d1b21ef9294bc8e4682d0baedfbf3f8020724c WatchSource:0}: Error finding container 88ad2ed36123d8d79effe85181d1b21ef9294bc8e4682d0baedfbf3f8020724c: Status 404 returned error can't find the container with id 88ad2ed36123d8d79effe85181d1b21ef9294bc8e4682d0baedfbf3f8020724c
Jul 15 17:01:15.675249 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.675226    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6w956"
Jul 15 17:01:15.792901 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:15.792874    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6w956"]
Jul 15 17:01:15.795584 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:01:15.795536    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3efe260_102a_456d_9c7d_a47277cf1f35.slice/crio-9fc39f7911d4e284fd654f85a82f033f15b3e7f83b69780865bb3b746ca1d27a WatchSource:0}: Error finding container 9fc39f7911d4e284fd654f85a82f033f15b3e7f83b69780865bb3b746ca1d27a: Status 404 returned error can't find the container with id 9fc39f7911d4e284fd654f85a82f033f15b3e7f83b69780865bb3b746ca1d27a
Jul 15 17:01:16.536576 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:16.536534    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58845c4f86-llrx9" event={"ID":"2c59b417-2f6d-4949-ac54-94e9889f8379","Type":"ContainerStarted","Data":"88ad2ed36123d8d79effe85181d1b21ef9294bc8e4682d0baedfbf3f8020724c"}
Jul 15 17:01:16.538261 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:16.538198    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6w956" event={"ID":"e3efe260-102a-456d-9c7d-a47277cf1f35","Type":"ContainerStarted","Data":"9fc39f7911d4e284fd654f85a82f033f15b3e7f83b69780865bb3b746ca1d27a"}
Jul 15 17:01:20.520422 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:01:20.520385    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-7kv7g" podUID="a62f9005-2c66-4d30-9102-121aae529be8"
Jul 15 17:01:21.521982 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:21.521955    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-f6q75"
Jul 15 17:01:21.559541 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:21.559509    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tlgvp" event={"ID":"fee99ffe-3358-4efa-8014-a66f6a973724","Type":"ContainerStarted","Data":"65e3ad615e5d6476e8fd831459c2e29a1450d98a8ee162c2c0222339d5975ae8"}
Jul 15 17:01:21.560719 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:21.560696    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6w956" event={"ID":"e3efe260-102a-456d-9c7d-a47277cf1f35","Type":"ContainerStarted","Data":"86c4ee246d970b88701358af739f3cb34d0c20cf22b77886a34d7a90b1bff54f"}
Jul 15 17:01:21.561172 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:21.561154    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6w956"
Jul 15 17:01:21.562276 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:21.562255    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58845c4f86-llrx9" event={"ID":"2c59b417-2f6d-4949-ac54-94e9889f8379","Type":"ContainerStarted","Data":"3ded6a6e9e69cc026caa6b8175a56475ee86ce5ed37c39fb0c4645ea06d9b06a"}
Jul 15 17:01:21.565969 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:21.565946    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6w956"
Jul 15 17:01:21.587387 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:21.587336    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6w956" podStartSLOduration=0.947570615 podStartE2EDuration="6.587318244s" totalImagesPullingTime="5.639747629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:01:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:21.585871982 +0000 UTC m=+297.603285103" watchObservedRunningTime="2026-07-15 17:01:21.587318244 +0000 UTC m=+297.604731341"
Jul 15 17:01:21.609784 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:21.609744    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58845c4f86-llrx9" podStartSLOduration=1.038500177 podStartE2EDuration="6.609728141s" totalImagesPullingTime="5.571227964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:01:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:21.609248738 +0000 UTC m=+297.626661858" watchObservedRunningTime="2026-07-15 17:01:21.609728141 +0000 UTC m=+297.627141237"
Jul 15 17:01:24.347142 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:24.347084    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 17:01:24.348581 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:24.348540    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 17:01:24.359392 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:24.359372    2575 kubelet.go:1648] "Image garbage collection succeeded"
Jul 15 17:01:25.475027 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:25.474996    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:25.483030 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:25.475094    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:25.483030 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:25.480303    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:25.578457 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:25.578428    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:01:27.127788 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:27.127770    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 17:01:27.583586 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:27.583534    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tlgvp" event={"ID":"fee99ffe-3358-4efa-8014-a66f6a973724","Type":"ContainerStarted","Data":"6ffd71a1ff3bf8817951f9aa16787c568daf565fddb48fcd7ad449d47692d982"}
Jul 15 17:01:30.595632 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:30.595600    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tlgvp" event={"ID":"fee99ffe-3358-4efa-8014-a66f6a973724","Type":"ContainerStarted","Data":"da99ccec2a4cc09a52cc00377e593f6df8bc69a56aed35503262285d59c1e7ec"}
Jul 15 17:01:30.616138 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:30.616094    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tlgvp" podStartSLOduration=33.185200785 podStartE2EDuration="48.616082443s" totalImagesPullingTime="15.430881658s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 17:00:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:30.61516944 +0000 UTC m=+306.632582537" watchObservedRunningTime="2026-07-15 17:01:30.616082443 +0000 UTC m=+306.633495539"
Jul 15 17:01:32.520116 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:32.520084    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 17:01:36.832709 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:36.832673    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8d7c5"]
Jul 15 17:01:36.836025 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:36.836009    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:36.838519 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:36.838496    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 15 17:01:36.838648 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:36.838496    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 15 17:01:36.839392 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:36.839375    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zdfnb\""
Jul 15 17:01:36.839468 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:36.839431    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 15 17:01:36.839468 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:36.839380    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 15 17:01:36.839468 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:36.839450    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 15 17:01:36.839676 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:36.839662    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 15 17:01:36.924716 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:36.924685    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-node-exporter-wtmp\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:36.924716 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:36.924718    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:36.924906 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:36.924743    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-node-exporter-accelerators-collector-config\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:36.924906 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:36.924820    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-node-exporter-textfile\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:36.924906 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:36.924853    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-node-exporter-tls\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:36.924906 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:36.924876    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-metrics-client-ca\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:36.924906 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:36.924903    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-root\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:36.925075 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:36.924970    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-sys\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:36.925075 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:36.925008    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vlpd\" (UniqueName: \"kubernetes.io/projected/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-kube-api-access-7vlpd\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:37.026260 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:37.026230    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-node-exporter-textfile\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:37.026260 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:37.026262    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-node-exporter-tls\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:37.026466 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:37.026284    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-metrics-client-ca\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:37.026466 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:37.026342    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-root\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:37.026466 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:37.026417    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-sys\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:37.026643 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:37.026472    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vlpd\" (UniqueName: \"kubernetes.io/projected/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-kube-api-access-7vlpd\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:37.026643 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:37.026504    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-root\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:37.026643 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:37.026518    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-node-exporter-wtmp\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:37.026643 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:37.026592    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:37.026643 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:37.026631    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-node-exporter-accelerators-collector-config\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:37.026882 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:37.026715    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-node-exporter-wtmp\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:37.026882 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:37.026595    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-sys\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:37.027007 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:37.026987    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-metrics-client-ca\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:37.027127 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:37.027107    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-node-exporter-textfile\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:37.027370 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:37.027348    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-node-exporter-accelerators-collector-config\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:37.028729 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:37.028704    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-node-exporter-tls\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:37.028839 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:37.028824    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:37.035035 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:37.035012    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vlpd\" (UniqueName: \"kubernetes.io/projected/b74b3ae7-7e36-4419-be70-b15a30fe8cf9-kube-api-access-7vlpd\") pod \"node-exporter-8d7c5\" (UID: \"b74b3ae7-7e36-4419-be70-b15a30fe8cf9\") " pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:37.145636 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:37.145612    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8d7c5"
Jul 15 17:01:37.156756 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:01:37.156719    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb74b3ae7_7e36_4419_be70_b15a30fe8cf9.slice/crio-af472f804ce400db7986fc47adc1e8e5ccc749e6666ee6651fe738a4cc303aee WatchSource:0}: Error finding container af472f804ce400db7986fc47adc1e8e5ccc749e6666ee6651fe738a4cc303aee: Status 404 returned error can't find the container with id af472f804ce400db7986fc47adc1e8e5ccc749e6666ee6651fe738a4cc303aee
Jul 15 17:01:37.619171 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:37.619136    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8d7c5" event={"ID":"b74b3ae7-7e36-4419-be70-b15a30fe8cf9","Type":"ContainerStarted","Data":"af472f804ce400db7986fc47adc1e8e5ccc749e6666ee6651fe738a4cc303aee"}
Jul 15 17:01:39.627409 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:39.627376    2575 generic.go:356] "Generic (PLEG): container finished" podID="b74b3ae7-7e36-4419-be70-b15a30fe8cf9" containerID="e3cac19ae65bd4df3154f3c2813171c975d73e67446226b36dcd08a836c9f543" exitCode=0
Jul 15 17:01:39.627806 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:39.627436    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8d7c5" event={"ID":"b74b3ae7-7e36-4419-be70-b15a30fe8cf9","Type":"ContainerDied","Data":"e3cac19ae65bd4df3154f3c2813171c975d73e67446226b36dcd08a836c9f543"}
Jul 15 17:01:40.632721 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:40.632684    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8d7c5" event={"ID":"b74b3ae7-7e36-4419-be70-b15a30fe8cf9","Type":"ContainerStarted","Data":"49a407422e9eefc70c9549d1f1a0d8475e0c84a6ea1cc85c0060163e44a130e0"}
Jul 15 17:01:40.633207 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:40.632738    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8d7c5" event={"ID":"b74b3ae7-7e36-4419-be70-b15a30fe8cf9","Type":"ContainerStarted","Data":"160c57fe2fa399061ac7b1ffa7a9a4e61c2a5900400b4235447e4be610d448f1"}
Jul 15 17:01:40.653327 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:40.653280    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8d7c5" podStartSLOduration=2.873666767 podStartE2EDuration="4.653266075s" totalImagesPullingTime="1.715775794s" totalInitContainerRuntime="63.823514ms" isStatefulPod=true podCreationTimestamp="2026-07-15 17:01:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:40.652437571 +0000 UTC m=+316.669850728" watchObservedRunningTime="2026-07-15 17:01:40.653266075 +0000 UTC m=+316.670679172"
Jul 15 17:01:46.521054 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:46.521010    2575 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s" podUID="9ce368e3-4392-458d-aeee-c8e06c545966" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 15 17:01:52.146429 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:52.146391    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58845c4f86-llrx9"]
Jul 15 17:01:56.520942 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:01:56.520898    2575 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s" podUID="9ce368e3-4392-458d-aeee-c8e06c545966" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 15 17:02:06.520798 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:06.520759    2575 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s" podUID="9ce368e3-4392-458d-aeee-c8e06c545966" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 15 17:02:06.527612 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:06.527589    2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s"
Jul 15 17:02:06.528193 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:06.528156    2575 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"6098d937f33442ed099e937eedfa8b6eae403c573c293375a94ba8608d811639"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 15 17:02:06.528250 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:06.528233    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s" podUID="9ce368e3-4392-458d-aeee-c8e06c545966" containerName="service-proxy" containerID="cri-o://6098d937f33442ed099e937eedfa8b6eae403c573c293375a94ba8608d811639" gracePeriod=30
Jul 15 17:02:06.713611 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:06.713584    2575 generic.go:356] "Generic (PLEG): container finished" podID="9ce368e3-4392-458d-aeee-c8e06c545966" containerID="6098d937f33442ed099e937eedfa8b6eae403c573c293375a94ba8608d811639" exitCode=2
Jul 15 17:02:06.713732 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:06.713658    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s" event={"ID":"9ce368e3-4392-458d-aeee-c8e06c545966","Type":"ContainerDied","Data":"6098d937f33442ed099e937eedfa8b6eae403c573c293375a94ba8608d811639"}
Jul 15 17:02:06.713732 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:06.713702    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5449f8fcb7-vqg9s" event={"ID":"9ce368e3-4392-458d-aeee-c8e06c545966","Type":"ContainerStarted","Data":"1fdb1e656987ecd14f819e702746485aa98383d1ec4d8f89342a14bb4193071c"}
Jul 15 17:02:17.165715 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.165624    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-58845c4f86-llrx9" podUID="2c59b417-2f6d-4949-ac54-94e9889f8379" containerName="console" containerID="cri-o://3ded6a6e9e69cc026caa6b8175a56475ee86ce5ed37c39fb0c4645ea06d9b06a" gracePeriod=15
Jul 15 17:02:17.403816 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.403796    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:02:17.505991 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.505927    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-service-ca\") pod \"2c59b417-2f6d-4949-ac54-94e9889f8379\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") "
Jul 15 17:02:17.505991 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.505963    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-oauth-serving-cert\") pod \"2c59b417-2f6d-4949-ac54-94e9889f8379\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") "
Jul 15 17:02:17.505991 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.505981    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2c59b417-2f6d-4949-ac54-94e9889f8379-console-serving-cert\") pod \"2c59b417-2f6d-4949-ac54-94e9889f8379\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") "
Jul 15 17:02:17.506207 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.506184    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-trusted-ca-bundle\") pod \"2c59b417-2f6d-4949-ac54-94e9889f8379\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") "
Jul 15 17:02:17.506247 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.506234    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gp2nq\" (UniqueName: \"kubernetes.io/projected/2c59b417-2f6d-4949-ac54-94e9889f8379-kube-api-access-gp2nq\") pod \"2c59b417-2f6d-4949-ac54-94e9889f8379\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") "
Jul 15 17:02:17.506347 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.506276    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2c59b417-2f6d-4949-ac54-94e9889f8379-console-oauth-config\") pod \"2c59b417-2f6d-4949-ac54-94e9889f8379\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") "
Jul 15 17:02:17.506347 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.506307    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-console-config\") pod \"2c59b417-2f6d-4949-ac54-94e9889f8379\" (UID: \"2c59b417-2f6d-4949-ac54-94e9889f8379\") "
Jul 15 17:02:17.506453 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.506402    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-service-ca" (OuterVolumeSpecName: "service-ca") pod "2c59b417-2f6d-4949-ac54-94e9889f8379" (UID: "2c59b417-2f6d-4949-ac54-94e9889f8379"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:02:17.506507 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.506446    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2c59b417-2f6d-4949-ac54-94e9889f8379" (UID: "2c59b417-2f6d-4949-ac54-94e9889f8379"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:02:17.506702 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.506668    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2c59b417-2f6d-4949-ac54-94e9889f8379" (UID: "2c59b417-2f6d-4949-ac54-94e9889f8379"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:02:17.506702 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.506682    2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-service-ca\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:02:17.506855 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.506721    2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-oauth-serving-cert\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:02:17.506855 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.506718    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-console-config" (OuterVolumeSpecName: "console-config") pod "2c59b417-2f6d-4949-ac54-94e9889f8379" (UID: "2c59b417-2f6d-4949-ac54-94e9889f8379"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:02:17.508276 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.508252    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c59b417-2f6d-4949-ac54-94e9889f8379-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2c59b417-2f6d-4949-ac54-94e9889f8379" (UID: "2c59b417-2f6d-4949-ac54-94e9889f8379"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:02:17.508341 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.508316    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c59b417-2f6d-4949-ac54-94e9889f8379-kube-api-access-gp2nq" (OuterVolumeSpecName: "kube-api-access-gp2nq") pod "2c59b417-2f6d-4949-ac54-94e9889f8379" (UID: "2c59b417-2f6d-4949-ac54-94e9889f8379"). InnerVolumeSpecName "kube-api-access-gp2nq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:02:17.508388 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.508369    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c59b417-2f6d-4949-ac54-94e9889f8379-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2c59b417-2f6d-4949-ac54-94e9889f8379" (UID: "2c59b417-2f6d-4949-ac54-94e9889f8379"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:02:17.607906 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.607876    2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-trusted-ca-bundle\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:02:17.607906 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.607897    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gp2nq\" (UniqueName: \"kubernetes.io/projected/2c59b417-2f6d-4949-ac54-94e9889f8379-kube-api-access-gp2nq\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:02:17.607906 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.607908    2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2c59b417-2f6d-4949-ac54-94e9889f8379-console-oauth-config\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:02:17.608157 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.607921    2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2c59b417-2f6d-4949-ac54-94e9889f8379-console-config\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:02:17.608157 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.607934    2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2c59b417-2f6d-4949-ac54-94e9889f8379-console-serving-cert\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:02:17.757071 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.756990    2575 generic.go:356] "Generic (PLEG): container finished" podID="2c59b417-2f6d-4949-ac54-94e9889f8379" containerID="3ded6a6e9e69cc026caa6b8175a56475ee86ce5ed37c39fb0c4645ea06d9b06a" exitCode=0
Jul 15 17:02:17.757071 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.757041    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58845c4f86-llrx9"
Jul 15 17:02:17.757071 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.757050    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58845c4f86-llrx9" event={"ID":"2c59b417-2f6d-4949-ac54-94e9889f8379","Type":"ContainerDied","Data":"3ded6a6e9e69cc026caa6b8175a56475ee86ce5ed37c39fb0c4645ea06d9b06a"}
Jul 15 17:02:17.757300 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.757124    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58845c4f86-llrx9" event={"ID":"2c59b417-2f6d-4949-ac54-94e9889f8379","Type":"ContainerDied","Data":"88ad2ed36123d8d79effe85181d1b21ef9294bc8e4682d0baedfbf3f8020724c"}
Jul 15 17:02:17.757300 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.757162    2575 scope.go:117] "RemoveContainer" containerID="3ded6a6e9e69cc026caa6b8175a56475ee86ce5ed37c39fb0c4645ea06d9b06a"
Jul 15 17:02:17.766623 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.766606    2575 scope.go:117] "RemoveContainer" containerID="3ded6a6e9e69cc026caa6b8175a56475ee86ce5ed37c39fb0c4645ea06d9b06a"
Jul 15 17:02:17.766907 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:02:17.766889    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ded6a6e9e69cc026caa6b8175a56475ee86ce5ed37c39fb0c4645ea06d9b06a\": container with ID starting with 3ded6a6e9e69cc026caa6b8175a56475ee86ce5ed37c39fb0c4645ea06d9b06a not found: ID does not exist" containerID="3ded6a6e9e69cc026caa6b8175a56475ee86ce5ed37c39fb0c4645ea06d9b06a"
Jul 15 17:02:17.766970 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.766915    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ded6a6e9e69cc026caa6b8175a56475ee86ce5ed37c39fb0c4645ea06d9b06a"} err="failed to get container status \"3ded6a6e9e69cc026caa6b8175a56475ee86ce5ed37c39fb0c4645ea06d9b06a\": rpc error: code = NotFound desc = could not find container \"3ded6a6e9e69cc026caa6b8175a56475ee86ce5ed37c39fb0c4645ea06d9b06a\": container with ID starting with 3ded6a6e9e69cc026caa6b8175a56475ee86ce5ed37c39fb0c4645ea06d9b06a not found: ID does not exist"
Jul 15 17:02:17.778609 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.778588    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58845c4f86-llrx9"]
Jul 15 17:02:17.782693 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:17.782653    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58845c4f86-llrx9"]
Jul 15 17:02:18.524279 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:18.524251    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c59b417-2f6d-4949-ac54-94e9889f8379" path="/var/lib/kubelet/pods/2c59b417-2f6d-4949-ac54-94e9889f8379/volumes"
Jul 15 17:02:37.342537 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:37.342492    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs\") pod \"network-metrics-daemon-7kv7g\" (UID: \"a62f9005-2c66-4d30-9102-121aae529be8\") " pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 17:02:37.344836 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:37.344814    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a62f9005-2c66-4d30-9102-121aae529be8-metrics-certs\") pod \"network-metrics-daemon-7kv7g\" (UID: \"a62f9005-2c66-4d30-9102-121aae529be8\") " pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 17:02:37.623460 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:37.623430    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pqk7d\""
Jul 15 17:02:37.631075 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:37.631053    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7kv7g"
Jul 15 17:02:37.752538 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:37.752514    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7kv7g"]
Jul 15 17:02:37.754984 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:02:37.754959    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda62f9005_2c66_4d30_9102_121aae529be8.slice/crio-5936b502b2d9f5b8891e83ea94723debc63a862ef0ad27596621da1863a8a77b WatchSource:0}: Error finding container 5936b502b2d9f5b8891e83ea94723debc63a862ef0ad27596621da1863a8a77b: Status 404 returned error can't find the container with id 5936b502b2d9f5b8891e83ea94723debc63a862ef0ad27596621da1863a8a77b
Jul 15 17:02:37.819052 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:37.819020    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7kv7g" event={"ID":"a62f9005-2c66-4d30-9102-121aae529be8","Type":"ContainerStarted","Data":"5936b502b2d9f5b8891e83ea94723debc63a862ef0ad27596621da1863a8a77b"}
Jul 15 17:02:38.828959 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:38.828926    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7kv7g" event={"ID":"a62f9005-2c66-4d30-9102-121aae529be8","Type":"ContainerStarted","Data":"7f165da369f2dd08b88c8c9688eb224e10edf393797e95f5b4e7846aa69e13dd"}
Jul 15 17:02:39.834199 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:39.834156    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7kv7g" event={"ID":"a62f9005-2c66-4d30-9102-121aae529be8","Type":"ContainerStarted","Data":"26264ff95f1ace145808baf0e90e66bcb8e147dcd0712fc1fddd788972318fbf"}
Jul 15 17:02:39.850013 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:02:39.849967    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7kv7g" podStartSLOduration=373.886871832 podStartE2EDuration="6m14.849953781s" totalImagesPullingTime="963.081949ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:02:39.849216764 +0000 UTC m=+375.866629884" watchObservedRunningTime="2026-07-15 17:02:39.849953781 +0000 UTC m=+375.867366913"
Jul 15 17:03:12.968444 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:12.968411    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7ffd9bcfc9-zz9cf"]
Jul 15 17:03:12.968870 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:12.968640    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c59b417-2f6d-4949-ac54-94e9889f8379" containerName="console"
Jul 15 17:03:12.968870 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:12.968650    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c59b417-2f6d-4949-ac54-94e9889f8379" containerName="console"
Jul 15 17:03:12.968870 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:12.968704    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c59b417-2f6d-4949-ac54-94e9889f8379" containerName="console"
Jul 15 17:03:12.973082 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:12.971786    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:12.975522 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:12.975502    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 15 17:03:12.975658 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:12.975502    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 15 17:03:12.976391 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:12.976371    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 15 17:03:12.976490 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:12.976423    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 15 17:03:12.976490 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:12.976424    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 15 17:03:12.976629 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:12.976513    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 15 17:03:12.976629 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:12.976602    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 15 17:03:12.976715 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:12.976636    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-d5smq\""
Jul 15 17:03:12.980478 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:12.980455    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 15 17:03:12.981350 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:12.981330    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ffd9bcfc9-zz9cf"]
Jul 15 17:03:13.081010 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.080979    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-console-config\") pod \"console-7ffd9bcfc9-zz9cf\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") " pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:13.081010 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.081016    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-oauth-serving-cert\") pod \"console-7ffd9bcfc9-zz9cf\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") " pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:13.081175 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.081034    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-service-ca\") pod \"console-7ffd9bcfc9-zz9cf\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") " pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:13.081175 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.081055    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-console-oauth-config\") pod \"console-7ffd9bcfc9-zz9cf\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") " pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:13.081175 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.081124    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl84g\" (UniqueName: \"kubernetes.io/projected/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-kube-api-access-kl84g\") pod \"console-7ffd9bcfc9-zz9cf\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") " pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:13.081270 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.081175    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-console-serving-cert\") pod \"console-7ffd9bcfc9-zz9cf\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") " pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:13.081270 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.081207    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-trusted-ca-bundle\") pod \"console-7ffd9bcfc9-zz9cf\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") " pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:13.182266 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.182228    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-console-config\") pod \"console-7ffd9bcfc9-zz9cf\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") " pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:13.182266 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.182266    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-oauth-serving-cert\") pod \"console-7ffd9bcfc9-zz9cf\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") " pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:13.182408 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.182282    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-service-ca\") pod \"console-7ffd9bcfc9-zz9cf\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") " pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:13.182408 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.182305    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-console-oauth-config\") pod \"console-7ffd9bcfc9-zz9cf\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") " pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:13.182614 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.182594    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl84g\" (UniqueName: \"kubernetes.io/projected/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-kube-api-access-kl84g\") pod \"console-7ffd9bcfc9-zz9cf\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") " pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:13.182667 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.182648    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-console-serving-cert\") pod \"console-7ffd9bcfc9-zz9cf\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") " pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:13.182714 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.182687    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-trusted-ca-bundle\") pod \"console-7ffd9bcfc9-zz9cf\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") " pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:13.183090 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.183062    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-service-ca\") pod \"console-7ffd9bcfc9-zz9cf\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") " pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:13.183189 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.183062    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-oauth-serving-cert\") pod \"console-7ffd9bcfc9-zz9cf\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") " pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:13.183189 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.183112    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-console-config\") pod \"console-7ffd9bcfc9-zz9cf\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") " pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:13.183346 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.183328    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-trusted-ca-bundle\") pod \"console-7ffd9bcfc9-zz9cf\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") " pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:13.184793 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.184774    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-console-oauth-config\") pod \"console-7ffd9bcfc9-zz9cf\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") " pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:13.185048 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.185029    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-console-serving-cert\") pod \"console-7ffd9bcfc9-zz9cf\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") " pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:13.191329 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.191308    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl84g\" (UniqueName: \"kubernetes.io/projected/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-kube-api-access-kl84g\") pod \"console-7ffd9bcfc9-zz9cf\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") " pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:13.283152 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.283081    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:13.404856 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.404826    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ffd9bcfc9-zz9cf"]
Jul 15 17:03:13.407890 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:03:13.407866    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62d4bd49_f0c6_4bd5_bebf_1e68b96d780c.slice/crio-0888714ea9280e9abcb999cdd33dd3a6829b8c41e524abce0a758af7712cbd12 WatchSource:0}: Error finding container 0888714ea9280e9abcb999cdd33dd3a6829b8c41e524abce0a758af7712cbd12: Status 404 returned error can't find the container with id 0888714ea9280e9abcb999cdd33dd3a6829b8c41e524abce0a758af7712cbd12
Jul 15 17:03:13.940302 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.940261    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ffd9bcfc9-zz9cf" event={"ID":"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c","Type":"ContainerStarted","Data":"7bc9b4f48a8cd127ac80b346e0785d7a35ff6db039b7e29d846ea3c7b91de8e7"}
Jul 15 17:03:13.940466 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.940318    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ffd9bcfc9-zz9cf" event={"ID":"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c","Type":"ContainerStarted","Data":"0888714ea9280e9abcb999cdd33dd3a6829b8c41e524abce0a758af7712cbd12"}
Jul 15 17:03:13.959096 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:13.959047    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7ffd9bcfc9-zz9cf" podStartSLOduration=1.959032176 podStartE2EDuration="1.959032176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:03:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:03:13.957832636 +0000 UTC m=+409.975245733" watchObservedRunningTime="2026-07-15 17:03:13.959032176 +0000 UTC m=+409.976445272"
Jul 15 17:03:23.284009 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:23.283975    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:23.284381 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:23.284036    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:23.288531 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:23.288509    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:03:23.979404 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:03:23.979374    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:05:32.370649 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:32.370614    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-67fcj"]
Jul 15 17:05:32.372870 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:32.372852    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-67fcj"
Jul 15 17:05:32.375948 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:32.375925    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 15 17:05:32.376779 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:32.376759    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-msdvl\""
Jul 15 17:05:32.376881 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:32.376763    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 15 17:05:32.392936 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:32.392914    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-67fcj"]
Jul 15 17:05:32.506912 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:32.506878    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7kmj\" (UniqueName: \"kubernetes.io/projected/77bc1858-8821-4fb9-a70b-61956adaa46e-kube-api-access-m7kmj\") pod \"openshift-lws-operator-fd8ccff4c-67fcj\" (UID: \"77bc1858-8821-4fb9-a70b-61956adaa46e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-67fcj"
Jul 15 17:05:32.506912 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:32.506917    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77bc1858-8821-4fb9-a70b-61956adaa46e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-67fcj\" (UID: \"77bc1858-8821-4fb9-a70b-61956adaa46e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-67fcj"
Jul 15 17:05:32.607973 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:32.607946    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7kmj\" (UniqueName: \"kubernetes.io/projected/77bc1858-8821-4fb9-a70b-61956adaa46e-kube-api-access-m7kmj\") pod \"openshift-lws-operator-fd8ccff4c-67fcj\" (UID: \"77bc1858-8821-4fb9-a70b-61956adaa46e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-67fcj"
Jul 15 17:05:32.608130 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:32.607983    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77bc1858-8821-4fb9-a70b-61956adaa46e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-67fcj\" (UID: \"77bc1858-8821-4fb9-a70b-61956adaa46e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-67fcj"
Jul 15 17:05:32.608308 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:32.608293    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77bc1858-8821-4fb9-a70b-61956adaa46e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-67fcj\" (UID: \"77bc1858-8821-4fb9-a70b-61956adaa46e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-67fcj"
Jul 15 17:05:32.616921 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:32.616899    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7kmj\" (UniqueName: \"kubernetes.io/projected/77bc1858-8821-4fb9-a70b-61956adaa46e-kube-api-access-m7kmj\") pod \"openshift-lws-operator-fd8ccff4c-67fcj\" (UID: \"77bc1858-8821-4fb9-a70b-61956adaa46e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-67fcj"
Jul 15 17:05:32.681956 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:32.681880    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-67fcj"
Jul 15 17:05:32.805057 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:32.804972    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-67fcj"]
Jul 15 17:05:32.807833 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:05:32.807809    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77bc1858_8821_4fb9_a70b_61956adaa46e.slice/crio-7156bec7065fcb1840fa088c1e9524bf6245acd0e637ee4d6f95d022e96ccee1 WatchSource:0}: Error finding container 7156bec7065fcb1840fa088c1e9524bf6245acd0e637ee4d6f95d022e96ccee1: Status 404 returned error can't find the container with id 7156bec7065fcb1840fa088c1e9524bf6245acd0e637ee4d6f95d022e96ccee1
Jul 15 17:05:33.385640 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:33.385609    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-67fcj" event={"ID":"77bc1858-8821-4fb9-a70b-61956adaa46e","Type":"ContainerStarted","Data":"7156bec7065fcb1840fa088c1e9524bf6245acd0e637ee4d6f95d022e96ccee1"}
Jul 15 17:05:35.394777 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:35.394732    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-67fcj" event={"ID":"77bc1858-8821-4fb9-a70b-61956adaa46e","Type":"ContainerStarted","Data":"98537fdc5b09451ac176ed5d80bafeac0df30f992a54ac1136b1ac78b8d0f819"}
Jul 15 17:05:35.424761 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:35.424701    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-67fcj" podStartSLOduration=1.033471496 podStartE2EDuration="3.424681313s" totalImagesPullingTime="2.391209817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:05:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:05:35.422612888 +0000 UTC m=+551.440025984" watchObservedRunningTime="2026-07-15 17:05:35.424681313 +0000 UTC m=+551.442094410"
Jul 15 17:05:51.101814 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.101765    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5"]
Jul 15 17:05:51.108793 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.108769    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5"
Jul 15 17:05:51.112783 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.112756    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 15 17:05:51.112903 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.112867    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 15 17:05:51.112960 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.112873    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5xhsk\""
Jul 15 17:05:51.113385 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.113372    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 15 17:05:51.113576 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.113550    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 15 17:05:51.122744 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.122724    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5"]
Jul 15 17:05:51.234355 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.234322    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hb5q\" (UniqueName: \"kubernetes.io/projected/df8d2fcf-ace8-4e7e-a47a-0f5342b1411d-kube-api-access-9hb5q\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-q4zv5\" (UID: \"df8d2fcf-ace8-4e7e-a47a-0f5342b1411d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5"
Jul 15 17:05:51.234514 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.234360    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df8d2fcf-ace8-4e7e-a47a-0f5342b1411d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-q4zv5\" (UID: \"df8d2fcf-ace8-4e7e-a47a-0f5342b1411d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5"
Jul 15 17:05:51.234514 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.234384    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df8d2fcf-ace8-4e7e-a47a-0f5342b1411d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-q4zv5\" (UID: \"df8d2fcf-ace8-4e7e-a47a-0f5342b1411d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5"
Jul 15 17:05:51.234514 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.234451    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df8d2fcf-ace8-4e7e-a47a-0f5342b1411d-tmp\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-q4zv5\" (UID: \"df8d2fcf-ace8-4e7e-a47a-0f5342b1411d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5"
Jul 15 17:05:51.234514 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.234506    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df8d2fcf-ace8-4e7e-a47a-0f5342b1411d-manifests\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-q4zv5\" (UID: \"df8d2fcf-ace8-4e7e-a47a-0f5342b1411d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5"
Jul 15 17:05:51.335651 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.335614    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hb5q\" (UniqueName: \"kubernetes.io/projected/df8d2fcf-ace8-4e7e-a47a-0f5342b1411d-kube-api-access-9hb5q\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-q4zv5\" (UID: \"df8d2fcf-ace8-4e7e-a47a-0f5342b1411d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5"
Jul 15 17:05:51.335651 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.335658    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df8d2fcf-ace8-4e7e-a47a-0f5342b1411d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-q4zv5\" (UID: \"df8d2fcf-ace8-4e7e-a47a-0f5342b1411d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5"
Jul 15 17:05:51.335880 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.335679    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df8d2fcf-ace8-4e7e-a47a-0f5342b1411d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-q4zv5\" (UID: \"df8d2fcf-ace8-4e7e-a47a-0f5342b1411d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5"
Jul 15 17:05:51.335880 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.335703    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df8d2fcf-ace8-4e7e-a47a-0f5342b1411d-tmp\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-q4zv5\" (UID: \"df8d2fcf-ace8-4e7e-a47a-0f5342b1411d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5"
Jul 15 17:05:51.335880 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.335736    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df8d2fcf-ace8-4e7e-a47a-0f5342b1411d-manifests\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-q4zv5\" (UID: \"df8d2fcf-ace8-4e7e-a47a-0f5342b1411d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5"
Jul 15 17:05:51.336340 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.336314    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df8d2fcf-ace8-4e7e-a47a-0f5342b1411d-manifests\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-q4zv5\" (UID: \"df8d2fcf-ace8-4e7e-a47a-0f5342b1411d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5"
Jul 15 17:05:51.338207 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.338181    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df8d2fcf-ace8-4e7e-a47a-0f5342b1411d-tmp\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-q4zv5\" (UID: \"df8d2fcf-ace8-4e7e-a47a-0f5342b1411d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5"
Jul 15 17:05:51.338306 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.338267    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df8d2fcf-ace8-4e7e-a47a-0f5342b1411d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-q4zv5\" (UID: \"df8d2fcf-ace8-4e7e-a47a-0f5342b1411d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5"
Jul 15 17:05:51.338404 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.338386    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df8d2fcf-ace8-4e7e-a47a-0f5342b1411d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-q4zv5\" (UID: \"df8d2fcf-ace8-4e7e-a47a-0f5342b1411d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5"
Jul 15 17:05:51.346976 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.346956    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hb5q\" (UniqueName: \"kubernetes.io/projected/df8d2fcf-ace8-4e7e-a47a-0f5342b1411d-kube-api-access-9hb5q\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-q4zv5\" (UID: \"df8d2fcf-ace8-4e7e-a47a-0f5342b1411d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5"
Jul 15 17:05:51.418872 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.418846    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5"
Jul 15 17:05:51.556627 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:51.556596    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5"]
Jul 15 17:05:51.559670 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:05:51.559641    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf8d2fcf_ace8_4e7e_a47a_0f5342b1411d.slice/crio-718ff67ea5aed17b3c1df9bad537e7c380352b65628a20875e253186500931c9 WatchSource:0}: Error finding container 718ff67ea5aed17b3c1df9bad537e7c380352b65628a20875e253186500931c9: Status 404 returned error can't find the container with id 718ff67ea5aed17b3c1df9bad537e7c380352b65628a20875e253186500931c9
Jul 15 17:05:52.456541 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:52.456504    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5" event={"ID":"df8d2fcf-ace8-4e7e-a47a-0f5342b1411d","Type":"ContainerStarted","Data":"718ff67ea5aed17b3c1df9bad537e7c380352b65628a20875e253186500931c9"}
Jul 15 17:05:57.474958 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:57.474920    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5" event={"ID":"df8d2fcf-ace8-4e7e-a47a-0f5342b1411d","Type":"ContainerStarted","Data":"cb808cc93627f9e278a346e3fef49d340d9ff8235cb702e422a3bd1711d8fd85"}
Jul 15 17:05:58.480720 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:58.480693    2575 generic.go:356] "Generic (PLEG): container finished" podID="df8d2fcf-ace8-4e7e-a47a-0f5342b1411d" containerID="cb808cc93627f9e278a346e3fef49d340d9ff8235cb702e422a3bd1711d8fd85" exitCode=0
Jul 15 17:05:58.480993 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:58.480740    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5" event={"ID":"df8d2fcf-ace8-4e7e-a47a-0f5342b1411d","Type":"ContainerDied","Data":"cb808cc93627f9e278a346e3fef49d340d9ff8235cb702e422a3bd1711d8fd85"}
Jul 15 17:05:59.486198 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:59.486164    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5" event={"ID":"df8d2fcf-ace8-4e7e-a47a-0f5342b1411d","Type":"ContainerStarted","Data":"8bcf4bd6fc450bb7aba262cde703db953da715c2ce46cfa34e520aa39bd0194a"}
Jul 15 17:05:59.486602 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:59.486305    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5"
Jul 15 17:05:59.507305 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:05:59.507258    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5" podStartSLOduration=1.250874831 podStartE2EDuration="8.50724265s" totalImagesPullingTime="6.227044233s" totalInitContainerRuntime="1.029323586s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:05:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:05:59.505663677 +0000 UTC m=+575.523076773" watchObservedRunningTime="2026-07-15 17:05:59.50724265 +0000 UTC m=+575.524655745"
Jul 15 17:06:10.492117 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:10.492087    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-q4zv5"
Jul 15 17:06:13.108923 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.108884    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-b44776b5f-m2gtp"]
Jul 15 17:06:13.116722 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.116695    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-b44776b5f-m2gtp"
Jul 15 17:06:13.119451 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.119425    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 15 17:06:13.119451 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.119444    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 15 17:06:13.119650 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.119469    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 15 17:06:13.119650 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.119500    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 15 17:06:13.122753 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.122733    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-b44776b5f-m2gtp"]
Jul 15 17:06:13.188998 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.188968    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjhs8\" (UniqueName: \"kubernetes.io/projected/7beab3a5-bfb9-435a-93c5-3ba8265d6ad0-kube-api-access-tjhs8\") pod \"kube-auth-proxy-b44776b5f-m2gtp\" (UID: \"7beab3a5-bfb9-435a-93c5-3ba8265d6ad0\") " pod="openshift-ingress/kube-auth-proxy-b44776b5f-m2gtp"
Jul 15 17:06:13.189122 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.189004    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7beab3a5-bfb9-435a-93c5-3ba8265d6ad0-tls-certs\") pod \"kube-auth-proxy-b44776b5f-m2gtp\" (UID: \"7beab3a5-bfb9-435a-93c5-3ba8265d6ad0\") " pod="openshift-ingress/kube-auth-proxy-b44776b5f-m2gtp"
Jul 15 17:06:13.189122 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.189033    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7beab3a5-bfb9-435a-93c5-3ba8265d6ad0-tmp\") pod \"kube-auth-proxy-b44776b5f-m2gtp\" (UID: \"7beab3a5-bfb9-435a-93c5-3ba8265d6ad0\") " pod="openshift-ingress/kube-auth-proxy-b44776b5f-m2gtp"
Jul 15 17:06:13.244674 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.244650    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-8zhqd"]
Jul 15 17:06:13.247540 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.247525    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-8zhqd"
Jul 15 17:06:13.249850 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.249835    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 15 17:06:13.250330 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.250316    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-rsmd9\""
Jul 15 17:06:13.259127 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.259108    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-8zhqd"]
Jul 15 17:06:13.290155 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.290132    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjhs8\" (UniqueName: \"kubernetes.io/projected/7beab3a5-bfb9-435a-93c5-3ba8265d6ad0-kube-api-access-tjhs8\") pod \"kube-auth-proxy-b44776b5f-m2gtp\" (UID: \"7beab3a5-bfb9-435a-93c5-3ba8265d6ad0\") " pod="openshift-ingress/kube-auth-proxy-b44776b5f-m2gtp"
Jul 15 17:06:13.290265 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.290165    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd6kg\" (UniqueName: \"kubernetes.io/projected/47c4c998-668f-4026-9f1d-95f3a398b7f9-kube-api-access-pd6kg\") pod \"odh-model-controller-7998c67dc4-8zhqd\" (UID: \"47c4c998-668f-4026-9f1d-95f3a398b7f9\") " pod="opendatahub/odh-model-controller-7998c67dc4-8zhqd"
Jul 15 17:06:13.290265 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.290186    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7beab3a5-bfb9-435a-93c5-3ba8265d6ad0-tls-certs\") pod \"kube-auth-proxy-b44776b5f-m2gtp\" (UID: \"7beab3a5-bfb9-435a-93c5-3ba8265d6ad0\") " pod="openshift-ingress/kube-auth-proxy-b44776b5f-m2gtp"
Jul 15 17:06:13.290265 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.290233    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7beab3a5-bfb9-435a-93c5-3ba8265d6ad0-tmp\") pod \"kube-auth-proxy-b44776b5f-m2gtp\" (UID: \"7beab3a5-bfb9-435a-93c5-3ba8265d6ad0\") " pod="openshift-ingress/kube-auth-proxy-b44776b5f-m2gtp"
Jul 15 17:06:13.290265 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:06:13.290258    2575 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 15 17:06:13.290460 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:06:13.290312    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7beab3a5-bfb9-435a-93c5-3ba8265d6ad0-tls-certs podName:7beab3a5-bfb9-435a-93c5-3ba8265d6ad0 nodeName:}" failed. No retries permitted until 2026-07-15 17:06:13.790298199 +0000 UTC m=+589.807711273 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7beab3a5-bfb9-435a-93c5-3ba8265d6ad0-tls-certs") pod "kube-auth-proxy-b44776b5f-m2gtp" (UID: "7beab3a5-bfb9-435a-93c5-3ba8265d6ad0") : secret "kube-auth-proxy-tls" not found
Jul 15 17:06:13.290460 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.290256    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47c4c998-668f-4026-9f1d-95f3a398b7f9-cert\") pod \"odh-model-controller-7998c67dc4-8zhqd\" (UID: \"47c4c998-668f-4026-9f1d-95f3a398b7f9\") " pod="opendatahub/odh-model-controller-7998c67dc4-8zhqd"
Jul 15 17:06:13.292270 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.292244    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7beab3a5-bfb9-435a-93c5-3ba8265d6ad0-tmp\") pod \"kube-auth-proxy-b44776b5f-m2gtp\" (UID: \"7beab3a5-bfb9-435a-93c5-3ba8265d6ad0\") " pod="openshift-ingress/kube-auth-proxy-b44776b5f-m2gtp"
Jul 15 17:06:13.298616 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.298592    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjhs8\" (UniqueName: \"kubernetes.io/projected/7beab3a5-bfb9-435a-93c5-3ba8265d6ad0-kube-api-access-tjhs8\") pod \"kube-auth-proxy-b44776b5f-m2gtp\" (UID: \"7beab3a5-bfb9-435a-93c5-3ba8265d6ad0\") " pod="openshift-ingress/kube-auth-proxy-b44776b5f-m2gtp"
Jul 15 17:06:13.391145 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.391116    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd6kg\" (UniqueName: \"kubernetes.io/projected/47c4c998-668f-4026-9f1d-95f3a398b7f9-kube-api-access-pd6kg\") pod \"odh-model-controller-7998c67dc4-8zhqd\" (UID: \"47c4c998-668f-4026-9f1d-95f3a398b7f9\") " pod="opendatahub/odh-model-controller-7998c67dc4-8zhqd"
Jul 15 17:06:13.391263 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.391169    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47c4c998-668f-4026-9f1d-95f3a398b7f9-cert\") pod \"odh-model-controller-7998c67dc4-8zhqd\" (UID: \"47c4c998-668f-4026-9f1d-95f3a398b7f9\") " pod="opendatahub/odh-model-controller-7998c67dc4-8zhqd"
Jul 15 17:06:13.391310 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:06:13.391284    2575 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 15 17:06:13.391358 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:06:13.391347    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47c4c998-668f-4026-9f1d-95f3a398b7f9-cert podName:47c4c998-668f-4026-9f1d-95f3a398b7f9 nodeName:}" failed. No retries permitted until 2026-07-15 17:06:13.891329698 +0000 UTC m=+589.908742773 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/47c4c998-668f-4026-9f1d-95f3a398b7f9-cert") pod "odh-model-controller-7998c67dc4-8zhqd" (UID: "47c4c998-668f-4026-9f1d-95f3a398b7f9") : secret "odh-model-controller-webhook-cert" not found
Jul 15 17:06:13.402000 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.401979    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd6kg\" (UniqueName: \"kubernetes.io/projected/47c4c998-668f-4026-9f1d-95f3a398b7f9-kube-api-access-pd6kg\") pod \"odh-model-controller-7998c67dc4-8zhqd\" (UID: \"47c4c998-668f-4026-9f1d-95f3a398b7f9\") " pod="opendatahub/odh-model-controller-7998c67dc4-8zhqd"
Jul 15 17:06:13.795403 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.795332    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7beab3a5-bfb9-435a-93c5-3ba8265d6ad0-tls-certs\") pod \"kube-auth-proxy-b44776b5f-m2gtp\" (UID: \"7beab3a5-bfb9-435a-93c5-3ba8265d6ad0\") " pod="openshift-ingress/kube-auth-proxy-b44776b5f-m2gtp"
Jul 15 17:06:13.797470 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.797452    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7beab3a5-bfb9-435a-93c5-3ba8265d6ad0-tls-certs\") pod \"kube-auth-proxy-b44776b5f-m2gtp\" (UID: \"7beab3a5-bfb9-435a-93c5-3ba8265d6ad0\") " pod="openshift-ingress/kube-auth-proxy-b44776b5f-m2gtp"
Jul 15 17:06:13.896359 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:13.896332    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47c4c998-668f-4026-9f1d-95f3a398b7f9-cert\") pod \"odh-model-controller-7998c67dc4-8zhqd\" (UID: \"47c4c998-668f-4026-9f1d-95f3a398b7f9\") " pod="opendatahub/odh-model-controller-7998c67dc4-8zhqd"
Jul 15 17:06:13.896512 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:06:13.896493    2575 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 15 17:06:13.896602 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:06:13.896590    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47c4c998-668f-4026-9f1d-95f3a398b7f9-cert podName:47c4c998-668f-4026-9f1d-95f3a398b7f9 nodeName:}" failed. No retries permitted until 2026-07-15 17:06:14.896548788 +0000 UTC m=+590.913961883 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/47c4c998-668f-4026-9f1d-95f3a398b7f9-cert") pod "odh-model-controller-7998c67dc4-8zhqd" (UID: "47c4c998-668f-4026-9f1d-95f3a398b7f9") : secret "odh-model-controller-webhook-cert" not found
Jul 15 17:06:14.027308 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.027281    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-b44776b5f-m2gtp"
Jul 15 17:06:14.164764 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.164729    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-b44776b5f-m2gtp"]
Jul 15 17:06:14.167575 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:06:14.167538    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7beab3a5_bfb9_435a_93c5_3ba8265d6ad0.slice/crio-52f3c6be06620ec46150748d0ca39874a2229c8bf3cc6be4047ec20662a86d7f WatchSource:0}: Error finding container 52f3c6be06620ec46150748d0ca39874a2229c8bf3cc6be4047ec20662a86d7f: Status 404 returned error can't find the container with id 52f3c6be06620ec46150748d0ca39874a2229c8bf3cc6be4047ec20662a86d7f
Jul 15 17:06:14.200288 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.200264    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-frrfv"]
Jul 15 17:06:14.204291 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.204276    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-frrfv"
Jul 15 17:06:14.206998 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.206980    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-sm5gh\""
Jul 15 17:06:14.207314 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.207297    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 15 17:06:14.221158 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.221136    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-frrfv"]
Jul 15 17:06:14.300419 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.300389    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx75n\" (UniqueName: \"kubernetes.io/projected/191790f8-4646-4b5f-af3e-667b4fbf9af2-kube-api-access-zx75n\") pod \"model-serving-api-54d89bd5fc-frrfv\" (UID: \"191790f8-4646-4b5f-af3e-667b4fbf9af2\") " pod="opendatahub/model-serving-api-54d89bd5fc-frrfv"
Jul 15 17:06:14.300549 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.300428    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/191790f8-4646-4b5f-af3e-667b4fbf9af2-tls-certs\") pod \"model-serving-api-54d89bd5fc-frrfv\" (UID: \"191790f8-4646-4b5f-af3e-667b4fbf9af2\") " pod="opendatahub/model-serving-api-54d89bd5fc-frrfv"
Jul 15 17:06:14.401436 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.401403    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx75n\" (UniqueName: \"kubernetes.io/projected/191790f8-4646-4b5f-af3e-667b4fbf9af2-kube-api-access-zx75n\") pod \"model-serving-api-54d89bd5fc-frrfv\" (UID: \"191790f8-4646-4b5f-af3e-667b4fbf9af2\") " pod="opendatahub/model-serving-api-54d89bd5fc-frrfv"
Jul 15 17:06:14.401641 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.401443    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/191790f8-4646-4b5f-af3e-667b4fbf9af2-tls-certs\") pod \"model-serving-api-54d89bd5fc-frrfv\" (UID: \"191790f8-4646-4b5f-af3e-667b4fbf9af2\") " pod="opendatahub/model-serving-api-54d89bd5fc-frrfv"
Jul 15 17:06:14.403878 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.403859    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/191790f8-4646-4b5f-af3e-667b4fbf9af2-tls-certs\") pod \"model-serving-api-54d89bd5fc-frrfv\" (UID: \"191790f8-4646-4b5f-af3e-667b4fbf9af2\") " pod="opendatahub/model-serving-api-54d89bd5fc-frrfv"
Jul 15 17:06:14.410527 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.410507    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx75n\" (UniqueName: \"kubernetes.io/projected/191790f8-4646-4b5f-af3e-667b4fbf9af2-kube-api-access-zx75n\") pod \"model-serving-api-54d89bd5fc-frrfv\" (UID: \"191790f8-4646-4b5f-af3e-667b4fbf9af2\") " pod="opendatahub/model-serving-api-54d89bd5fc-frrfv"
Jul 15 17:06:14.513795 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.513763    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-frrfv"
Jul 15 17:06:14.537373 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.537349    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-b44776b5f-m2gtp" event={"ID":"7beab3a5-bfb9-435a-93c5-3ba8265d6ad0","Type":"ContainerStarted","Data":"52f3c6be06620ec46150748d0ca39874a2229c8bf3cc6be4047ec20662a86d7f"}
Jul 15 17:06:14.647215 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.647192    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-frrfv"]
Jul 15 17:06:14.649608 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:06:14.649582    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod191790f8_4646_4b5f_af3e_667b4fbf9af2.slice/crio-9dcdc358d99197d615516f15bbb6622f526165aace6d482a7a95c8d1d2bd0a4c WatchSource:0}: Error finding container 9dcdc358d99197d615516f15bbb6622f526165aace6d482a7a95c8d1d2bd0a4c: Status 404 returned error can't find the container with id 9dcdc358d99197d615516f15bbb6622f526165aace6d482a7a95c8d1d2bd0a4c
Jul 15 17:06:14.658012 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.657990    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"]
Jul 15 17:06:14.662183 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.662166    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:14.665210 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.665192    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 15 17:06:14.665210 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.665204    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-8dsqb\""
Jul 15 17:06:14.665210 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.665215    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 15 17:06:14.665412 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.665238    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 15 17:06:14.665412 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.665324    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 15 17:06:14.675264 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.675242    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"]
Jul 15 17:06:14.703180 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.703152    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-w4r56\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:14.703270 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.703203    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b2988f3-6c7f-4ff4-b4ff-af799d898736-local-certs\") pod \"istiod-openshift-gateway-7574459656-w4r56\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:14.703270 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.703227    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-w4r56\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:14.703270 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.703245    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt52c\" (UniqueName: \"kubernetes.io/projected/5b2988f3-6c7f-4ff4-b4ff-af799d898736-kube-api-access-dt52c\") pod \"istiod-openshift-gateway-7574459656-w4r56\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:14.703376 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.703281    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-w4r56\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:14.703376 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.703314    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5b2988f3-6c7f-4ff4-b4ff-af799d898736-cacerts\") pod \"istiod-openshift-gateway-7574459656-w4r56\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:14.703376 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.703336    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-token\") pod \"istiod-openshift-gateway-7574459656-w4r56\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:14.804684 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.804653    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b2988f3-6c7f-4ff4-b4ff-af799d898736-local-certs\") pod \"istiod-openshift-gateway-7574459656-w4r56\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:14.804864 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.804693    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-w4r56\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:14.804864 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.804717    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt52c\" (UniqueName: \"kubernetes.io/projected/5b2988f3-6c7f-4ff4-b4ff-af799d898736-kube-api-access-dt52c\") pod \"istiod-openshift-gateway-7574459656-w4r56\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:14.804864 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.804746    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-w4r56\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:14.804864 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.804786    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5b2988f3-6c7f-4ff4-b4ff-af799d898736-cacerts\") pod \"istiod-openshift-gateway-7574459656-w4r56\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:14.804864 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.804813    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-token\") pod \"istiod-openshift-gateway-7574459656-w4r56\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:14.805109 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.804866    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-w4r56\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:14.805459 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.805428    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-w4r56\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:14.807052 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.807027    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b2988f3-6c7f-4ff4-b4ff-af799d898736-local-certs\") pod \"istiod-openshift-gateway-7574459656-w4r56\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:14.807171 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.807156    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-w4r56\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:14.807212 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.807186    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-w4r56\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:14.807246 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.807212    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5b2988f3-6c7f-4ff4-b4ff-af799d898736-cacerts\") pod \"istiod-openshift-gateway-7574459656-w4r56\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:14.812487 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.812467    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-token\") pod \"istiod-openshift-gateway-7574459656-w4r56\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:14.812913 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.812896    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt52c\" (UniqueName: \"kubernetes.io/projected/5b2988f3-6c7f-4ff4-b4ff-af799d898736-kube-api-access-dt52c\") pod \"istiod-openshift-gateway-7574459656-w4r56\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:14.906179 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.906081    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47c4c998-668f-4026-9f1d-95f3a398b7f9-cert\") pod \"odh-model-controller-7998c67dc4-8zhqd\" (UID: \"47c4c998-668f-4026-9f1d-95f3a398b7f9\") " pod="opendatahub/odh-model-controller-7998c67dc4-8zhqd"
Jul 15 17:06:14.908595 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.908546    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47c4c998-668f-4026-9f1d-95f3a398b7f9-cert\") pod \"odh-model-controller-7998c67dc4-8zhqd\" (UID: \"47c4c998-668f-4026-9f1d-95f3a398b7f9\") " pod="opendatahub/odh-model-controller-7998c67dc4-8zhqd"
Jul 15 17:06:14.971464 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:14.971434    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:15.056948 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:15.056887    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-8zhqd"
Jul 15 17:06:15.104905 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:15.104857    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"]
Jul 15 17:06:15.107993 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:06:15.107966    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b2988f3_6c7f_4ff4_b4ff_af799d898736.slice/crio-4ae44b1d519b574469f08cb0f1d5e08cd5aaefa8b629aa6c2c90f3d8393bcfeb WatchSource:0}: Error finding container 4ae44b1d519b574469f08cb0f1d5e08cd5aaefa8b629aa6c2c90f3d8393bcfeb: Status 404 returned error can't find the container with id 4ae44b1d519b574469f08cb0f1d5e08cd5aaefa8b629aa6c2c90f3d8393bcfeb
Jul 15 17:06:15.198454 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:15.198372    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-8zhqd"]
Jul 15 17:06:15.201942 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:06:15.201912    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47c4c998_668f_4026_9f1d_95f3a398b7f9.slice/crio-a4b23c9a2cbc8619b96e1613ac612287935c29659249ba1fdd3ebec68f3a6ffe WatchSource:0}: Error finding container a4b23c9a2cbc8619b96e1613ac612287935c29659249ba1fdd3ebec68f3a6ffe: Status 404 returned error can't find the container with id a4b23c9a2cbc8619b96e1613ac612287935c29659249ba1fdd3ebec68f3a6ffe
Jul 15 17:06:15.541815 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:15.541725    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-frrfv" event={"ID":"191790f8-4646-4b5f-af3e-667b4fbf9af2","Type":"ContainerStarted","Data":"9dcdc358d99197d615516f15bbb6622f526165aace6d482a7a95c8d1d2bd0a4c"}
Jul 15 17:06:15.542798 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:15.542780    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-8zhqd" event={"ID":"47c4c998-668f-4026-9f1d-95f3a398b7f9","Type":"ContainerStarted","Data":"a4b23c9a2cbc8619b96e1613ac612287935c29659249ba1fdd3ebec68f3a6ffe"}
Jul 15 17:06:15.543908 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:15.543891    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56" event={"ID":"5b2988f3-6c7f-4ff4-b4ff-af799d898736","Type":"ContainerStarted","Data":"4ae44b1d519b574469f08cb0f1d5e08cd5aaefa8b629aa6c2c90f3d8393bcfeb"}
Jul 15 17:06:17.554061 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:17.553972    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-b44776b5f-m2gtp" event={"ID":"7beab3a5-bfb9-435a-93c5-3ba8265d6ad0","Type":"ContainerStarted","Data":"30f1f6592f7f4102721889dd75117880060e412fd4fd244c6c20ea9bc84d4321"}
Jul 15 17:06:21.633695 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:21.633654    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:06:21.634069 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:21.633719    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:06:22.442935 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.442876    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-b44776b5f-m2gtp" podStartSLOduration=6.415770071 podStartE2EDuration="9.442852788s" totalImagesPullingTime="3.027082717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:06:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:06:17.571070704 +0000 UTC m=+593.588483800" watchObservedRunningTime="2026-07-15 17:06:22.442852788 +0000 UTC m=+598.460265887"
Jul 15 17:06:22.444156 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.444128    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"]
Jul 15 17:06:22.447321 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.447300    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.449892 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.449872    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-vhc2n\""
Jul 15 17:06:22.471432 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.471407    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"]
Jul 15 17:06:22.472980 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.472960    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ea1850f-81f0-4321-be0d-fcbe74517f5d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.473082 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.472994    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ea1850f-81f0-4321-be0d-fcbe74517f5d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.473082 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.473018    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ea1850f-81f0-4321-be0d-fcbe74517f5d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.473151 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.473079    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ea1850f-81f0-4321-be0d-fcbe74517f5d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.473151 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.473106    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf49t\" (UniqueName: \"kubernetes.io/projected/7ea1850f-81f0-4321-be0d-fcbe74517f5d-kube-api-access-kf49t\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.473219 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.473194    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ea1850f-81f0-4321-be0d-fcbe74517f5d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.473270 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.473256    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ea1850f-81f0-4321-be0d-fcbe74517f5d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.473304 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.473289    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ea1850f-81f0-4321-be0d-fcbe74517f5d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.473337 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.473319    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ea1850f-81f0-4321-be0d-fcbe74517f5d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.573817 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.573784    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ea1850f-81f0-4321-be0d-fcbe74517f5d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.574006 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.573824    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ea1850f-81f0-4321-be0d-fcbe74517f5d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.574006 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.573850    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ea1850f-81f0-4321-be0d-fcbe74517f5d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.574006 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.573892    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ea1850f-81f0-4321-be0d-fcbe74517f5d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.574006 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.573931    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ea1850f-81f0-4321-be0d-fcbe74517f5d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.574006 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.573964    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ea1850f-81f0-4321-be0d-fcbe74517f5d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.574264 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.574031    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ea1850f-81f0-4321-be0d-fcbe74517f5d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.574264 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.574208    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ea1850f-81f0-4321-be0d-fcbe74517f5d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.574377 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.574259    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ea1850f-81f0-4321-be0d-fcbe74517f5d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.574377 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.574322    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ea1850f-81f0-4321-be0d-fcbe74517f5d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.574377 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.574329    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ea1850f-81f0-4321-be0d-fcbe74517f5d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.574530 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.574423    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf49t\" (UniqueName: \"kubernetes.io/projected/7ea1850f-81f0-4321-be0d-fcbe74517f5d-kube-api-access-kf49t\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.574530 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.574465    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ea1850f-81f0-4321-be0d-fcbe74517f5d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.574901 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.574880    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ea1850f-81f0-4321-be0d-fcbe74517f5d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.576672 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.576652    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ea1850f-81f0-4321-be0d-fcbe74517f5d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.576828 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.576810    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ea1850f-81f0-4321-be0d-fcbe74517f5d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.577164 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.577139    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-frrfv" event={"ID":"191790f8-4646-4b5f-af3e-667b4fbf9af2","Type":"ContainerStarted","Data":"1d88cf5a37663118e47dab5c6d284ccc5282949b7d931838e0120bafac784618"}
Jul 15 17:06:22.577301 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.577288    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-frrfv"
Jul 15 17:06:22.578597 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.578552    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-8zhqd" event={"ID":"47c4c998-668f-4026-9f1d-95f3a398b7f9","Type":"ContainerStarted","Data":"fdf8dac600802db757aaa8870da413cb59b3c41856c1cd9601aa967e56078670"}
Jul 15 17:06:22.578703 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.578623    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-8zhqd"
Jul 15 17:06:22.580025 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.580007    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56" event={"ID":"5b2988f3-6c7f-4ff4-b4ff-af799d898736","Type":"ContainerStarted","Data":"78bcf46c5bd11d9aaba83a62ad01f79b1a73c7bcd449f6b508e813622d1312f1"}
Jul 15 17:06:22.580212 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.580196    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:22.582171 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.582133    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:22.583001 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.582980    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf49t\" (UniqueName: \"kubernetes.io/projected/7ea1850f-81f0-4321-be0d-fcbe74517f5d-kube-api-access-kf49t\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.583179 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.583161    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ea1850f-81f0-4321-be0d-fcbe74517f5d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xd8ck\" (UID: \"7ea1850f-81f0-4321-be0d-fcbe74517f5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.605015 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.604964    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-frrfv" podStartSLOduration=1.619888305 podStartE2EDuration="8.604948053s" totalImagesPullingTime="6.985059748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:06:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:06:22.603864627 +0000 UTC m=+598.621277742" watchObservedRunningTime="2026-07-15 17:06:22.604948053 +0000 UTC m=+598.622361171"
Jul 15 17:06:22.626872 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.626824    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-8zhqd" podStartSLOduration=3.142884115 podStartE2EDuration="9.626806778s" totalImagesPullingTime="6.483922663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:06:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:06:22.626196456 +0000 UTC m=+598.643609553" watchObservedRunningTime="2026-07-15 17:06:22.626806778 +0000 UTC m=+598.644219889"
Jul 15 17:06:22.665398 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.665339    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56" podStartSLOduration=2.143633726 podStartE2EDuration="8.665297902s" totalImagesPullingTime="6.521664176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:06:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:06:22.664799252 +0000 UTC m=+598.682212349" watchObservedRunningTime="2026-07-15 17:06:22.665297902 +0000 UTC m=+598.682711000"
Jul 15 17:06:22.756537 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.756461    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:22.929203 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:22.929175    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"]
Jul 15 17:06:22.929694 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:06:22.929670    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ea1850f_81f0_4321_be0d_fcbe74517f5d.slice/crio-b0fbc224a4cc45409c4267cbdd9aff36d767bed93c389ad7b9edfcc44f6f161f WatchSource:0}: Error finding container b0fbc224a4cc45409c4267cbdd9aff36d767bed93c389ad7b9edfcc44f6f161f: Status 404 returned error can't find the container with id b0fbc224a4cc45409c4267cbdd9aff36d767bed93c389ad7b9edfcc44f6f161f
Jul 15 17:06:23.585481 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:23.585442    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck" event={"ID":"7ea1850f-81f0-4321-be0d-fcbe74517f5d","Type":"ContainerStarted","Data":"b0fbc224a4cc45409c4267cbdd9aff36d767bed93c389ad7b9edfcc44f6f161f"}
Jul 15 17:06:24.327975 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.327942    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4hfx2"]
Jul 15 17:06:24.331905 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.331880    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4hfx2"
Jul 15 17:06:24.335329 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.335306    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 15 17:06:24.335920 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.335900    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-sjmd2\""
Jul 15 17:06:24.344538 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.344509    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-f8zrm"]
Jul 15 17:06:24.349340 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.349319    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f8zrm"
Jul 15 17:06:24.349715 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.349695    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4hfx2"]
Jul 15 17:06:24.352722 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.352701    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 15 17:06:24.353308 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.353284    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-hjk4d\""
Jul 15 17:06:24.372197 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.372169    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-f8zrm"]
Jul 15 17:06:24.388045 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.388017    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pgs8\" (UniqueName: \"kubernetes.io/projected/2cc8746d-b197-452a-98d2-e89febe73f6c-kube-api-access-9pgs8\") pod \"kserve-controller-manager-d6f4b65b8-4hfx2\" (UID: \"2cc8746d-b197-452a-98d2-e89febe73f6c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4hfx2"
Jul 15 17:06:24.388186 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.388054    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f996b798-29d1-4cca-85fe-e8394d89ce9b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-f8zrm\" (UID: \"f996b798-29d1-4cca-85fe-e8394d89ce9b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f8zrm"
Jul 15 17:06:24.388186 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.388121    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2cc8746d-b197-452a-98d2-e89febe73f6c-cert\") pod \"kserve-controller-manager-d6f4b65b8-4hfx2\" (UID: \"2cc8746d-b197-452a-98d2-e89febe73f6c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4hfx2"
Jul 15 17:06:24.388301 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.388242    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54fhb\" (UniqueName: \"kubernetes.io/projected/f996b798-29d1-4cca-85fe-e8394d89ce9b-kube-api-access-54fhb\") pod \"llmisvc-controller-manager-777c6bd7d5-f8zrm\" (UID: \"f996b798-29d1-4cca-85fe-e8394d89ce9b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f8zrm"
Jul 15 17:06:24.393979 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.393956    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 17:06:24.394798 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.394779    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 17:06:24.489069 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.489034    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54fhb\" (UniqueName: \"kubernetes.io/projected/f996b798-29d1-4cca-85fe-e8394d89ce9b-kube-api-access-54fhb\") pod \"llmisvc-controller-manager-777c6bd7d5-f8zrm\" (UID: \"f996b798-29d1-4cca-85fe-e8394d89ce9b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f8zrm"
Jul 15 17:06:24.489069 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.489072    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pgs8\" (UniqueName: \"kubernetes.io/projected/2cc8746d-b197-452a-98d2-e89febe73f6c-kube-api-access-9pgs8\") pod \"kserve-controller-manager-d6f4b65b8-4hfx2\" (UID: \"2cc8746d-b197-452a-98d2-e89febe73f6c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4hfx2"
Jul 15 17:06:24.489327 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.489093    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f996b798-29d1-4cca-85fe-e8394d89ce9b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-f8zrm\" (UID: \"f996b798-29d1-4cca-85fe-e8394d89ce9b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f8zrm"
Jul 15 17:06:24.489327 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.489137    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2cc8746d-b197-452a-98d2-e89febe73f6c-cert\") pod \"kserve-controller-manager-d6f4b65b8-4hfx2\" (UID: \"2cc8746d-b197-452a-98d2-e89febe73f6c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4hfx2"
Jul 15 17:06:24.489327 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:06:24.489244    2575 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 15 17:06:24.489327 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:06:24.489295    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cc8746d-b197-452a-98d2-e89febe73f6c-cert podName:2cc8746d-b197-452a-98d2-e89febe73f6c nodeName:}" failed. No retries permitted until 2026-07-15 17:06:24.989277277 +0000 UTC m=+601.006690351 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2cc8746d-b197-452a-98d2-e89febe73f6c-cert") pod "kserve-controller-manager-d6f4b65b8-4hfx2" (UID: "2cc8746d-b197-452a-98d2-e89febe73f6c") : secret "kserve-webhook-server-cert" not found
Jul 15 17:06:24.492335 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.492310    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f996b798-29d1-4cca-85fe-e8394d89ce9b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-f8zrm\" (UID: \"f996b798-29d1-4cca-85fe-e8394d89ce9b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f8zrm"
Jul 15 17:06:24.507404 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.507376    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54fhb\" (UniqueName: \"kubernetes.io/projected/f996b798-29d1-4cca-85fe-e8394d89ce9b-kube-api-access-54fhb\") pod \"llmisvc-controller-manager-777c6bd7d5-f8zrm\" (UID: \"f996b798-29d1-4cca-85fe-e8394d89ce9b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f8zrm"
Jul 15 17:06:24.517364 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.517345    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pgs8\" (UniqueName: \"kubernetes.io/projected/2cc8746d-b197-452a-98d2-e89febe73f6c-kube-api-access-9pgs8\") pod \"kserve-controller-manager-d6f4b65b8-4hfx2\" (UID: \"2cc8746d-b197-452a-98d2-e89febe73f6c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4hfx2"
Jul 15 17:06:24.663490 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.663445    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f8zrm"
Jul 15 17:06:24.838018 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.837971    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-f8zrm"]
Jul 15 17:06:24.930061 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.929991    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"]
Jul 15 17:06:24.994705 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.994666    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2cc8746d-b197-452a-98d2-e89febe73f6c-cert\") pod \"kserve-controller-manager-d6f4b65b8-4hfx2\" (UID: \"2cc8746d-b197-452a-98d2-e89febe73f6c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4hfx2"
Jul 15 17:06:24.997177 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:24.997153    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2cc8746d-b197-452a-98d2-e89febe73f6c-cert\") pod \"kserve-controller-manager-d6f4b65b8-4hfx2\" (UID: \"2cc8746d-b197-452a-98d2-e89febe73f6c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4hfx2"
Jul 15 17:06:25.250624 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:25.250535    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4hfx2"
Jul 15 17:06:25.594629 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:25.594470    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56" podUID="5b2988f3-6c7f-4ff4-b4ff-af799d898736" containerName="discovery" containerID="cri-o://78bcf46c5bd11d9aaba83a62ad01f79b1a73c7bcd449f6b508e813622d1312f1" gracePeriod=30
Jul 15 17:06:25.738708 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:06:25.738655    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf996b798_29d1_4cca_85fe_e8394d89ce9b.slice/crio-8ca92f240bd998260afcf81f5a83ed531bbc5c9301abf6629e48a324b3cae7f6 WatchSource:0}: Error finding container 8ca92f240bd998260afcf81f5a83ed531bbc5c9301abf6629e48a324b3cae7f6: Status 404 returned error can't find the container with id 8ca92f240bd998260afcf81f5a83ed531bbc5c9301abf6629e48a324b3cae7f6
Jul 15 17:06:25.809504 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:25.809427    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:06:25.809693 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:25.809517    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:06:25.809693 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:25.809581    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:06:25.901927 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:25.901887    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4hfx2"]
Jul 15 17:06:25.903946 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:25.903926    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:25.905307 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:06:25.905287    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cc8746d_b197_452a_98d2_e89febe73f6c.slice/crio-76f7298e97f90247a0755745f4e3879ac6a6fba5c7e837e551f360a147b7bbd6 WatchSource:0}: Error finding container 76f7298e97f90247a0755745f4e3879ac6a6fba5c7e837e551f360a147b7bbd6: Status 404 returned error can't find the container with id 76f7298e97f90247a0755745f4e3879ac6a6fba5c7e837e551f360a147b7bbd6
Jul 15 17:06:26.003013 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.002986    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-csr-ca-configmap\") pod \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") "
Jul 15 17:06:26.003013 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.003017    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5b2988f3-6c7f-4ff4-b4ff-af799d898736-cacerts\") pod \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") "
Jul 15 17:06:26.003205 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.003037    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dt52c\" (UniqueName: \"kubernetes.io/projected/5b2988f3-6c7f-4ff4-b4ff-af799d898736-kube-api-access-dt52c\") pod \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") "
Jul 15 17:06:26.003205 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.003089    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b2988f3-6c7f-4ff4-b4ff-af799d898736-local-certs\") pod \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") "
Jul 15 17:06:26.003205 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.003120    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-kubeconfig\") pod \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") "
Jul 15 17:06:26.003205 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.003146    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-csr-dns-cert\") pod \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") "
Jul 15 17:06:26.003205 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.003175    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-token\") pod \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\" (UID: \"5b2988f3-6c7f-4ff4-b4ff-af799d898736\") "
Jul 15 17:06:26.003455 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.003427    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "5b2988f3-6c7f-4ff4-b4ff-af799d898736" (UID: "5b2988f3-6c7f-4ff4-b4ff-af799d898736"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:06:26.005823 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.005797    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b2988f3-6c7f-4ff4-b4ff-af799d898736-local-certs" (OuterVolumeSpecName: "local-certs") pod "5b2988f3-6c7f-4ff4-b4ff-af799d898736" (UID: "5b2988f3-6c7f-4ff4-b4ff-af799d898736"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:06:26.006001 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.005961    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b2988f3-6c7f-4ff4-b4ff-af799d898736-kube-api-access-dt52c" (OuterVolumeSpecName: "kube-api-access-dt52c") pod "5b2988f3-6c7f-4ff4-b4ff-af799d898736" (UID: "5b2988f3-6c7f-4ff4-b4ff-af799d898736"). InnerVolumeSpecName "kube-api-access-dt52c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:06:26.006078 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.006014    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-token" (OuterVolumeSpecName: "istio-token") pod "5b2988f3-6c7f-4ff4-b4ff-af799d898736" (UID: "5b2988f3-6c7f-4ff4-b4ff-af799d898736"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:06:26.006078 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.006033    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "5b2988f3-6c7f-4ff4-b4ff-af799d898736" (UID: "5b2988f3-6c7f-4ff4-b4ff-af799d898736"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:06:26.006182 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.006126    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "5b2988f3-6c7f-4ff4-b4ff-af799d898736" (UID: "5b2988f3-6c7f-4ff4-b4ff-af799d898736"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:06:26.006221 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.006181    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b2988f3-6c7f-4ff4-b4ff-af799d898736-cacerts" (OuterVolumeSpecName: "cacerts") pod "5b2988f3-6c7f-4ff4-b4ff-af799d898736" (UID: "5b2988f3-6c7f-4ff4-b4ff-af799d898736"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:06:26.104260 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.104190    2575 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-kubeconfig\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:06:26.104260 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.104214    2575 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-csr-dns-cert\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:06:26.104260 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.104223    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-token\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:06:26.104260 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.104232    2575 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5b2988f3-6c7f-4ff4-b4ff-af799d898736-istio-csr-ca-configmap\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:06:26.104260 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.104242    2575 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5b2988f3-6c7f-4ff4-b4ff-af799d898736-cacerts\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:06:26.104260 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.104250    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dt52c\" (UniqueName: \"kubernetes.io/projected/5b2988f3-6c7f-4ff4-b4ff-af799d898736-kube-api-access-dt52c\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:06:26.104260 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.104258    2575 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b2988f3-6c7f-4ff4-b4ff-af799d898736-local-certs\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:06:26.599298 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.599262    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f8zrm" event={"ID":"f996b798-29d1-4cca-85fe-e8394d89ce9b","Type":"ContainerStarted","Data":"8ca92f240bd998260afcf81f5a83ed531bbc5c9301abf6629e48a324b3cae7f6"}
Jul 15 17:06:26.600903 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.600872    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck" event={"ID":"7ea1850f-81f0-4321-be0d-fcbe74517f5d","Type":"ContainerStarted","Data":"f96044128de7b2fb37653a8bfdbec4efd067d732b945358931264d25bb6c2869"}
Jul 15 17:06:26.601944 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.601918    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4hfx2" event={"ID":"2cc8746d-b197-452a-98d2-e89febe73f6c","Type":"ContainerStarted","Data":"76f7298e97f90247a0755745f4e3879ac6a6fba5c7e837e551f360a147b7bbd6"}
Jul 15 17:06:26.603026 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.603003    2575 generic.go:356] "Generic (PLEG): container finished" podID="5b2988f3-6c7f-4ff4-b4ff-af799d898736" containerID="78bcf46c5bd11d9aaba83a62ad01f79b1a73c7bcd449f6b508e813622d1312f1" exitCode=0
Jul 15 17:06:26.603103 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.603035    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"
Jul 15 17:06:26.603103 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.603054    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56" event={"ID":"5b2988f3-6c7f-4ff4-b4ff-af799d898736","Type":"ContainerDied","Data":"78bcf46c5bd11d9aaba83a62ad01f79b1a73c7bcd449f6b508e813622d1312f1"}
Jul 15 17:06:26.603103 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.603092    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w4r56" event={"ID":"5b2988f3-6c7f-4ff4-b4ff-af799d898736","Type":"ContainerDied","Data":"4ae44b1d519b574469f08cb0f1d5e08cd5aaefa8b629aa6c2c90f3d8393bcfeb"}
Jul 15 17:06:26.603220 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.603119    2575 scope.go:117] "RemoveContainer" containerID="78bcf46c5bd11d9aaba83a62ad01f79b1a73c7bcd449f6b508e813622d1312f1"
Jul 15 17:06:26.615019 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.614997    2575 scope.go:117] "RemoveContainer" containerID="78bcf46c5bd11d9aaba83a62ad01f79b1a73c7bcd449f6b508e813622d1312f1"
Jul 15 17:06:26.615323 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:06:26.615304    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78bcf46c5bd11d9aaba83a62ad01f79b1a73c7bcd449f6b508e813622d1312f1\": container with ID starting with 78bcf46c5bd11d9aaba83a62ad01f79b1a73c7bcd449f6b508e813622d1312f1 not found: ID does not exist" containerID="78bcf46c5bd11d9aaba83a62ad01f79b1a73c7bcd449f6b508e813622d1312f1"
Jul 15 17:06:26.615392 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.615331    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78bcf46c5bd11d9aaba83a62ad01f79b1a73c7bcd449f6b508e813622d1312f1"} err="failed to get container status \"78bcf46c5bd11d9aaba83a62ad01f79b1a73c7bcd449f6b508e813622d1312f1\": rpc error: code = NotFound desc = could not find container \"78bcf46c5bd11d9aaba83a62ad01f79b1a73c7bcd449f6b508e813622d1312f1\": container with ID starting with 78bcf46c5bd11d9aaba83a62ad01f79b1a73c7bcd449f6b508e813622d1312f1 not found: ID does not exist"
Jul 15 17:06:26.629872 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.629827    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck" podStartSLOduration=1.75240551 podStartE2EDuration="4.629812014s" totalImagesPullingTime="2.877406504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:06:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:06:26.629592683 +0000 UTC m=+602.647005806" watchObservedRunningTime="2026-07-15 17:06:26.629812014 +0000 UTC m=+602.647225111"
Jul 15 17:06:26.647326 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.647294    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"]
Jul 15 17:06:26.651308 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.651276    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-w4r56"]
Jul 15 17:06:26.757663 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.757601    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:26.761972 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:26.761949    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:27.607687 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:27.607653    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:27.608506 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:27.608485    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xd8ck"
Jul 15 17:06:28.528035 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:28.527931    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b2988f3-6c7f-4ff4-b4ff-af799d898736" path="/var/lib/kubelet/pods/5b2988f3-6c7f-4ff4-b4ff-af799d898736/volumes"
Jul 15 17:06:29.617158 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:29.617124    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f8zrm" event={"ID":"f996b798-29d1-4cca-85fe-e8394d89ce9b","Type":"ContainerStarted","Data":"6d1d3a101f527d2b729a295c18ec3cb1431975916a5645c4ee84e93650dcbb7d"}
Jul 15 17:06:29.617528 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:29.617315    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f8zrm"
Jul 15 17:06:29.698525 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:29.698476    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f8zrm" podStartSLOduration=2.203288751 podStartE2EDuration="5.69846309s" totalImagesPullingTime="3.495174339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:06:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:06:29.697013103 +0000 UTC m=+605.714426200" watchObservedRunningTime="2026-07-15 17:06:29.69846309 +0000 UTC m=+605.715876183"
Jul 15 17:06:33.588299 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:33.587697    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-8zhqd"
Jul 15 17:06:33.592874 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:33.592845    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-frrfv"
Jul 15 17:06:34.641438 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:34.641399    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4hfx2" event={"ID":"2cc8746d-b197-452a-98d2-e89febe73f6c","Type":"ContainerStarted","Data":"77b5b897cfe2526392212f1d9a2eb1c8d75311f1ab7f2d148f25b5c1bb8c7a0c"}
Jul 15 17:06:34.641884 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:34.641493    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4hfx2"
Jul 15 17:06:34.658351 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:06:34.658305    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4hfx2" podStartSLOduration=2.554024842 podStartE2EDuration="10.658288971s" totalImagesPullingTime="8.104264129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:06:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:06:34.658008643 +0000 UTC m=+610.675421760" watchObservedRunningTime="2026-07-15 17:06:34.658288971 +0000 UTC m=+610.675702068"
Jul 15 17:07:00.623676 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:00.623597    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f8zrm"
Jul 15 17:07:05.649874 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:05.649846    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4hfx2"
Jul 15 17:07:14.025227 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.025195    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-smr6m"]
Jul 15 17:07:14.025604 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.025499    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b2988f3-6c7f-4ff4-b4ff-af799d898736" containerName="discovery"
Jul 15 17:07:14.025604 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.025512    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b2988f3-6c7f-4ff4-b4ff-af799d898736" containerName="discovery"
Jul 15 17:07:14.025604 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.025590    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b2988f3-6c7f-4ff4-b4ff-af799d898736" containerName="discovery"
Jul 15 17:07:14.042280 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.042251    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-smr6m"]
Jul 15 17:07:14.042413 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.042344    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-smr6m"
Jul 15 17:07:14.044648 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.044620    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 15 17:07:14.045495 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.045474    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 15 17:07:14.045679 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.045642    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-qx7l5\""
Jul 15 17:07:14.084349 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.084321    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22z6w\" (UniqueName: \"kubernetes.io/projected/fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8-kube-api-access-22z6w\") pod \"kuadrant-operator-catalog-smr6m\" (UID: \"fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8\") " pod="kuadrant-system/kuadrant-operator-catalog-smr6m"
Jul 15 17:07:14.184732 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.184699    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22z6w\" (UniqueName: \"kubernetes.io/projected/fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8-kube-api-access-22z6w\") pod \"kuadrant-operator-catalog-smr6m\" (UID: \"fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8\") " pod="kuadrant-system/kuadrant-operator-catalog-smr6m"
Jul 15 17:07:14.193759 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.193734    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22z6w\" (UniqueName: \"kubernetes.io/projected/fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8-kube-api-access-22z6w\") pod \"kuadrant-operator-catalog-smr6m\" (UID: \"fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8\") " pod="kuadrant-system/kuadrant-operator-catalog-smr6m"
Jul 15 17:07:14.352540 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.352446    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-smr6m"
Jul 15 17:07:14.382442 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.382411    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-smr6m"]
Jul 15 17:07:14.479741 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.479718    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-smr6m"]
Jul 15 17:07:14.482424 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:07:14.482400    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd3dd6bf_39f0_440b_b597_fc5cdc8fb0c8.slice/crio-0a3dc5f01dca535eb6d49ea79280f6a8f0b4fd56c6c8509a4452548ec0672d23 WatchSource:0}: Error finding container 0a3dc5f01dca535eb6d49ea79280f6a8f0b4fd56c6c8509a4452548ec0672d23: Status 404 returned error can't find the container with id 0a3dc5f01dca535eb6d49ea79280f6a8f0b4fd56c6c8509a4452548ec0672d23
Jul 15 17:07:14.484224 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.484206    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 17:07:14.594007 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.593968    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bpn7c"]
Jul 15 17:07:14.598371 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.598349    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bpn7c"
Jul 15 17:07:14.608082 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.608058    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bpn7c"]
Jul 15 17:07:14.688179 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.688146    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f4cw\" (UniqueName: \"kubernetes.io/projected/c111ce03-5c9b-4c21-b67d-89a7c081a198-kube-api-access-6f4cw\") pod \"kuadrant-operator-catalog-bpn7c\" (UID: \"c111ce03-5c9b-4c21-b67d-89a7c081a198\") " pod="kuadrant-system/kuadrant-operator-catalog-bpn7c"
Jul 15 17:07:14.789135 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.789097    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f4cw\" (UniqueName: \"kubernetes.io/projected/c111ce03-5c9b-4c21-b67d-89a7c081a198-kube-api-access-6f4cw\") pod \"kuadrant-operator-catalog-bpn7c\" (UID: \"c111ce03-5c9b-4c21-b67d-89a7c081a198\") " pod="kuadrant-system/kuadrant-operator-catalog-bpn7c"
Jul 15 17:07:14.796621 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.796594    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-smr6m" event={"ID":"fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8","Type":"ContainerStarted","Data":"0a3dc5f01dca535eb6d49ea79280f6a8f0b4fd56c6c8509a4452548ec0672d23"}
Jul 15 17:07:14.797455 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.797432    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f4cw\" (UniqueName: \"kubernetes.io/projected/c111ce03-5c9b-4c21-b67d-89a7c081a198-kube-api-access-6f4cw\") pod \"kuadrant-operator-catalog-bpn7c\" (UID: \"c111ce03-5c9b-4c21-b67d-89a7c081a198\") " pod="kuadrant-system/kuadrant-operator-catalog-bpn7c"
Jul 15 17:07:14.907527 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:14.907497    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bpn7c"
Jul 15 17:07:15.037570 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:15.037535    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bpn7c"]
Jul 15 17:07:15.039841 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:07:15.039814    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc111ce03_5c9b_4c21_b67d_89a7c081a198.slice/crio-5b42e78418c56ac16c9d5d78da3fa7e964a016ffbf2ffc4cb3cc03cacc5153fb WatchSource:0}: Error finding container 5b42e78418c56ac16c9d5d78da3fa7e964a016ffbf2ffc4cb3cc03cacc5153fb: Status 404 returned error can't find the container with id 5b42e78418c56ac16c9d5d78da3fa7e964a016ffbf2ffc4cb3cc03cacc5153fb
Jul 15 17:07:15.802752 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:15.802720    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bpn7c" event={"ID":"c111ce03-5c9b-4c21-b67d-89a7c081a198","Type":"ContainerStarted","Data":"5b42e78418c56ac16c9d5d78da3fa7e964a016ffbf2ffc4cb3cc03cacc5153fb"}
Jul 15 17:07:16.809152 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:16.809061    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-smr6m" event={"ID":"fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8","Type":"ContainerStarted","Data":"d28c80adb14a76a535eb7d8f18fd058c827d9b5d527c1f7e492c85404557f03a"}
Jul 15 17:07:16.809152 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:16.809130    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-smr6m" podUID="fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8" containerName="registry-server" containerID="cri-o://d28c80adb14a76a535eb7d8f18fd058c827d9b5d527c1f7e492c85404557f03a" gracePeriod=2
Jul 15 17:07:16.810733 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:16.810711    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bpn7c" event={"ID":"c111ce03-5c9b-4c21-b67d-89a7c081a198","Type":"ContainerStarted","Data":"d3f327921a0e590ffd2bc7ae4313113316be04be68b364cea833e9623f13f31f"}
Jul 15 17:07:16.827693 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:16.827648    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-smr6m" podStartSLOduration=0.746340573 podStartE2EDuration="2.827634686s" totalImagesPullingTime="2.081294113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:07:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:07:16.826622742 +0000 UTC m=+652.844035860" watchObservedRunningTime="2026-07-15 17:07:16.827634686 +0000 UTC m=+652.845047781"
Jul 15 17:07:16.843524 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:16.843487    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-bpn7c" podStartSLOduration=1.318657253 podStartE2EDuration="2.843473649s" totalImagesPullingTime="1.524816396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:07:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:07:16.842035016 +0000 UTC m=+652.859448137" watchObservedRunningTime="2026-07-15 17:07:16.843473649 +0000 UTC m=+652.860886745"
Jul 15 17:07:17.055708 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:17.055680    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-smr6m"
Jul 15 17:07:17.108387 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:17.108323    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22z6w\" (UniqueName: \"kubernetes.io/projected/fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8-kube-api-access-22z6w\") pod \"fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8\" (UID: \"fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8\") "
Jul 15 17:07:17.110381 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:17.110361    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8-kube-api-access-22z6w" (OuterVolumeSpecName: "kube-api-access-22z6w") pod "fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8" (UID: "fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8"). InnerVolumeSpecName "kube-api-access-22z6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:07:17.208856 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:17.208826    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22z6w\" (UniqueName: \"kubernetes.io/projected/fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8-kube-api-access-22z6w\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:07:17.816027 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:17.815998    2575 generic.go:356] "Generic (PLEG): container finished" podID="fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8" containerID="d28c80adb14a76a535eb7d8f18fd058c827d9b5d527c1f7e492c85404557f03a" exitCode=0
Jul 15 17:07:17.816438 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:17.816038    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-smr6m"
Jul 15 17:07:17.816438 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:17.816074    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-smr6m" event={"ID":"fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8","Type":"ContainerDied","Data":"d28c80adb14a76a535eb7d8f18fd058c827d9b5d527c1f7e492c85404557f03a"}
Jul 15 17:07:17.816438 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:17.816111    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-smr6m" event={"ID":"fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8","Type":"ContainerDied","Data":"0a3dc5f01dca535eb6d49ea79280f6a8f0b4fd56c6c8509a4452548ec0672d23"}
Jul 15 17:07:17.816438 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:17.816130    2575 scope.go:117] "RemoveContainer" containerID="d28c80adb14a76a535eb7d8f18fd058c827d9b5d527c1f7e492c85404557f03a"
Jul 15 17:07:17.824757 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:17.824738    2575 scope.go:117] "RemoveContainer" containerID="d28c80adb14a76a535eb7d8f18fd058c827d9b5d527c1f7e492c85404557f03a"
Jul 15 17:07:17.825000 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:07:17.824980    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d28c80adb14a76a535eb7d8f18fd058c827d9b5d527c1f7e492c85404557f03a\": container with ID starting with d28c80adb14a76a535eb7d8f18fd058c827d9b5d527c1f7e492c85404557f03a not found: ID does not exist" containerID="d28c80adb14a76a535eb7d8f18fd058c827d9b5d527c1f7e492c85404557f03a"
Jul 15 17:07:17.825062 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:17.825016    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d28c80adb14a76a535eb7d8f18fd058c827d9b5d527c1f7e492c85404557f03a"} err="failed to get container status \"d28c80adb14a76a535eb7d8f18fd058c827d9b5d527c1f7e492c85404557f03a\": rpc error: code = NotFound desc = could not find container \"d28c80adb14a76a535eb7d8f18fd058c827d9b5d527c1f7e492c85404557f03a\": container with ID starting with d28c80adb14a76a535eb7d8f18fd058c827d9b5d527c1f7e492c85404557f03a not found: ID does not exist"
Jul 15 17:07:17.836095 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:17.836036    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-smr6m"]
Jul 15 17:07:17.839602 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:17.839580    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-smr6m"]
Jul 15 17:07:18.524524 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:18.524494    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8" path="/var/lib/kubelet/pods/fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8/volumes"
Jul 15 17:07:24.907615 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:24.907584    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-bpn7c"
Jul 15 17:07:24.907993 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:24.907651    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-bpn7c"
Jul 15 17:07:24.928983 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:24.928959    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-bpn7c"
Jul 15 17:07:25.869644 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:07:25.869617    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-bpn7c"
Jul 15 17:08:02.459482 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:02.459448    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"]
Jul 15 17:08:02.459991 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:02.459746    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8" containerName="registry-server"
Jul 15 17:08:02.459991 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:02.459757    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8" containerName="registry-server"
Jul 15 17:08:02.459991 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:02.459808    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd3dd6bf-39f0-440b-b597-fc5cdc8fb0c8" containerName="registry-server"
Jul 15 17:08:02.467320 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:02.467298    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:02.469784 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:02.469758    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-984tg\""
Jul 15 17:08:02.477845 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:02.477820    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"]
Jul 15 17:08:02.548123 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:02.548090    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c225c69-7a29-4c0e-9f4c-fcc69030dbff-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-js9pj\" (UID: \"5c225c69-7a29-4c0e-9f4c-fcc69030dbff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:02.548279 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:02.548140    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f99x5\" (UniqueName: \"kubernetes.io/projected/5c225c69-7a29-4c0e-9f4c-fcc69030dbff-kube-api-access-f99x5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-js9pj\" (UID: \"5c225c69-7a29-4c0e-9f4c-fcc69030dbff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:02.649039 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:02.649010    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f99x5\" (UniqueName: \"kubernetes.io/projected/5c225c69-7a29-4c0e-9f4c-fcc69030dbff-kube-api-access-f99x5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-js9pj\" (UID: \"5c225c69-7a29-4c0e-9f4c-fcc69030dbff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:02.649143 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:02.649066    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c225c69-7a29-4c0e-9f4c-fcc69030dbff-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-js9pj\" (UID: \"5c225c69-7a29-4c0e-9f4c-fcc69030dbff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:02.649386 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:02.649370    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c225c69-7a29-4c0e-9f4c-fcc69030dbff-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-js9pj\" (UID: \"5c225c69-7a29-4c0e-9f4c-fcc69030dbff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:02.659784 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:02.659761    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f99x5\" (UniqueName: \"kubernetes.io/projected/5c225c69-7a29-4c0e-9f4c-fcc69030dbff-kube-api-access-f99x5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-js9pj\" (UID: \"5c225c69-7a29-4c0e-9f4c-fcc69030dbff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:02.777302 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:02.777235    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:02.905988 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:02.905964    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"]
Jul 15 17:08:02.908188 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:08:02.908164    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c225c69_7a29_4c0e_9f4c_fcc69030dbff.slice/crio-da4d43bcf8123ae6c2acbae15dde42daab63aabf15337ee46bbc98267ab947a8 WatchSource:0}: Error finding container da4d43bcf8123ae6c2acbae15dde42daab63aabf15337ee46bbc98267ab947a8: Status 404 returned error can't find the container with id da4d43bcf8123ae6c2acbae15dde42daab63aabf15337ee46bbc98267ab947a8
Jul 15 17:08:02.997123 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:02.997090    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj" event={"ID":"5c225c69-7a29-4c0e-9f4c-fcc69030dbff","Type":"ContainerStarted","Data":"da4d43bcf8123ae6c2acbae15dde42daab63aabf15337ee46bbc98267ab947a8"}
Jul 15 17:08:04.367999 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:04.367967    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-f7dfk"]
Jul 15 17:08:04.374217 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:04.374187    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f7dfk"
Jul 15 17:08:04.376787 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:04.376765    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 15 17:08:04.376943 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:04.376849    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-d89vc\""
Jul 15 17:08:04.382975 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:04.382952    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-f7dfk"]
Jul 15 17:08:04.464082 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:04.464014    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mrrv\" (UniqueName: \"kubernetes.io/projected/376e83c4-799d-4cb4-9d9c-062c949edc8e-kube-api-access-6mrrv\") pod \"dns-operator-controller-manager-864dd78c4c-f7dfk\" (UID: \"376e83c4-799d-4cb4-9d9c-062c949edc8e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f7dfk"
Jul 15 17:08:04.564752 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:04.564726    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mrrv\" (UniqueName: \"kubernetes.io/projected/376e83c4-799d-4cb4-9d9c-062c949edc8e-kube-api-access-6mrrv\") pod \"dns-operator-controller-manager-864dd78c4c-f7dfk\" (UID: \"376e83c4-799d-4cb4-9d9c-062c949edc8e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f7dfk"
Jul 15 17:08:04.575946 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:04.575918    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mrrv\" (UniqueName: \"kubernetes.io/projected/376e83c4-799d-4cb4-9d9c-062c949edc8e-kube-api-access-6mrrv\") pod \"dns-operator-controller-manager-864dd78c4c-f7dfk\" (UID: \"376e83c4-799d-4cb4-9d9c-062c949edc8e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f7dfk"
Jul 15 17:08:04.685173 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:04.685144    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f7dfk"
Jul 15 17:08:04.812002 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:04.811979    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-f7dfk"]
Jul 15 17:08:04.814532 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:08:04.814497    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod376e83c4_799d_4cb4_9d9c_062c949edc8e.slice/crio-61dbac30f6f77e7761d9b34c4ba52fff04889b0463fcb3bc0061fcce174508d2 WatchSource:0}: Error finding container 61dbac30f6f77e7761d9b34c4ba52fff04889b0463fcb3bc0061fcce174508d2: Status 404 returned error can't find the container with id 61dbac30f6f77e7761d9b34c4ba52fff04889b0463fcb3bc0061fcce174508d2
Jul 15 17:08:05.006279 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:05.006193    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f7dfk" event={"ID":"376e83c4-799d-4cb4-9d9c-062c949edc8e","Type":"ContainerStarted","Data":"61dbac30f6f77e7761d9b34c4ba52fff04889b0463fcb3bc0061fcce174508d2"}
Jul 15 17:08:07.017826 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:07.017793    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f7dfk" event={"ID":"376e83c4-799d-4cb4-9d9c-062c949edc8e","Type":"ContainerStarted","Data":"1d92be81b5c1fd4d392b8995eec8a941d0063df4ee65af902228c031d2abe884"}
Jul 15 17:08:07.018189 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:07.017923    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f7dfk"
Jul 15 17:08:07.036615 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:07.036544    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f7dfk" podStartSLOduration=0.949749736 podStartE2EDuration="3.036529222s" totalImagesPullingTime="2.086779486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:08:07.035715491 +0000 UTC m=+703.053128606" watchObservedRunningTime="2026-07-15 17:08:07.036529222 +0000 UTC m=+703.053942394"
Jul 15 17:08:07.333606 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:07.333513    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"]
Jul 15 17:08:07.336739 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:07.336723    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"
Jul 15 17:08:07.339713 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:07.339688    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-c5hhh\""
Jul 15 17:08:07.348547 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:07.348525    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"]
Jul 15 17:08:07.487573 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:07.487522    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gkpx\" (UniqueName: \"kubernetes.io/projected/7b92d62d-3a24-4262-bab3-e8b799adbbfe-kube-api-access-7gkpx\") pod \"limitador-operator-controller-manager-859d7b78bd-6d2ph\" (UID: \"7b92d62d-3a24-4262-bab3-e8b799adbbfe\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"
Jul 15 17:08:07.588721 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:07.588650    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gkpx\" (UniqueName: \"kubernetes.io/projected/7b92d62d-3a24-4262-bab3-e8b799adbbfe-kube-api-access-7gkpx\") pod \"limitador-operator-controller-manager-859d7b78bd-6d2ph\" (UID: \"7b92d62d-3a24-4262-bab3-e8b799adbbfe\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"
Jul 15 17:08:07.598629 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:07.598580    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gkpx\" (UniqueName: \"kubernetes.io/projected/7b92d62d-3a24-4262-bab3-e8b799adbbfe-kube-api-access-7gkpx\") pod \"limitador-operator-controller-manager-859d7b78bd-6d2ph\" (UID: \"7b92d62d-3a24-4262-bab3-e8b799adbbfe\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"
Jul 15 17:08:07.648423 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:07.648391    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"
Jul 15 17:08:07.787511 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:07.787481    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"]
Jul 15 17:08:07.790510 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:08:07.790479    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b92d62d_3a24_4262_bab3_e8b799adbbfe.slice/crio-ae1f240e88bf7843ceb2b78cdeb9c572953ef7fcb25d5aae8903f3e82a92eeff WatchSource:0}: Error finding container ae1f240e88bf7843ceb2b78cdeb9c572953ef7fcb25d5aae8903f3e82a92eeff: Status 404 returned error can't find the container with id ae1f240e88bf7843ceb2b78cdeb9c572953ef7fcb25d5aae8903f3e82a92eeff
Jul 15 17:08:07.940175 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:07.940146    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f6cd4fd5c-csqmm"]
Jul 15 17:08:07.943539 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:07.943522    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:07.955479 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:07.955441    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f6cd4fd5c-csqmm"]
Jul 15 17:08:08.024015 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.023978    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph" event={"ID":"7b92d62d-3a24-4262-bab3-e8b799adbbfe","Type":"ContainerStarted","Data":"ae1f240e88bf7843ceb2b78cdeb9c572953ef7fcb25d5aae8903f3e82a92eeff"}
Jul 15 17:08:08.093156 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.093113    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe433fb2-8705-4c66-bc23-4cae16489508-console-serving-cert\") pod \"console-7f6cd4fd5c-csqmm\" (UID: \"fe433fb2-8705-4c66-bc23-4cae16489508\") " pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:08.093312 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.093174    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe433fb2-8705-4c66-bc23-4cae16489508-service-ca\") pod \"console-7f6cd4fd5c-csqmm\" (UID: \"fe433fb2-8705-4c66-bc23-4cae16489508\") " pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:08.093312 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.093248    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe433fb2-8705-4c66-bc23-4cae16489508-console-oauth-config\") pod \"console-7f6cd4fd5c-csqmm\" (UID: \"fe433fb2-8705-4c66-bc23-4cae16489508\") " pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:08.093312 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.093275    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe433fb2-8705-4c66-bc23-4cae16489508-oauth-serving-cert\") pod \"console-7f6cd4fd5c-csqmm\" (UID: \"fe433fb2-8705-4c66-bc23-4cae16489508\") " pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:08.093312 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.093303    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr8sx\" (UniqueName: \"kubernetes.io/projected/fe433fb2-8705-4c66-bc23-4cae16489508-kube-api-access-kr8sx\") pod \"console-7f6cd4fd5c-csqmm\" (UID: \"fe433fb2-8705-4c66-bc23-4cae16489508\") " pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:08.093486 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.093361    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe433fb2-8705-4c66-bc23-4cae16489508-trusted-ca-bundle\") pod \"console-7f6cd4fd5c-csqmm\" (UID: \"fe433fb2-8705-4c66-bc23-4cae16489508\") " pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:08.093486 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.093454    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe433fb2-8705-4c66-bc23-4cae16489508-console-config\") pod \"console-7f6cd4fd5c-csqmm\" (UID: \"fe433fb2-8705-4c66-bc23-4cae16489508\") " pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:08.194488 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.194409    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr8sx\" (UniqueName: \"kubernetes.io/projected/fe433fb2-8705-4c66-bc23-4cae16489508-kube-api-access-kr8sx\") pod \"console-7f6cd4fd5c-csqmm\" (UID: \"fe433fb2-8705-4c66-bc23-4cae16489508\") " pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:08.194488 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.194458    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe433fb2-8705-4c66-bc23-4cae16489508-trusted-ca-bundle\") pod \"console-7f6cd4fd5c-csqmm\" (UID: \"fe433fb2-8705-4c66-bc23-4cae16489508\") " pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:08.194488 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.194476    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe433fb2-8705-4c66-bc23-4cae16489508-console-config\") pod \"console-7f6cd4fd5c-csqmm\" (UID: \"fe433fb2-8705-4c66-bc23-4cae16489508\") " pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:08.194776 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.194498    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe433fb2-8705-4c66-bc23-4cae16489508-console-serving-cert\") pod \"console-7f6cd4fd5c-csqmm\" (UID: \"fe433fb2-8705-4c66-bc23-4cae16489508\") " pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:08.194776 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.194547    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe433fb2-8705-4c66-bc23-4cae16489508-service-ca\") pod \"console-7f6cd4fd5c-csqmm\" (UID: \"fe433fb2-8705-4c66-bc23-4cae16489508\") " pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:08.194776 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.194615    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe433fb2-8705-4c66-bc23-4cae16489508-console-oauth-config\") pod \"console-7f6cd4fd5c-csqmm\" (UID: \"fe433fb2-8705-4c66-bc23-4cae16489508\") " pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:08.194776 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.194639    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe433fb2-8705-4c66-bc23-4cae16489508-oauth-serving-cert\") pod \"console-7f6cd4fd5c-csqmm\" (UID: \"fe433fb2-8705-4c66-bc23-4cae16489508\") " pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:08.195262 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.195240    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe433fb2-8705-4c66-bc23-4cae16489508-console-config\") pod \"console-7f6cd4fd5c-csqmm\" (UID: \"fe433fb2-8705-4c66-bc23-4cae16489508\") " pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:08.195369 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.195299    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe433fb2-8705-4c66-bc23-4cae16489508-oauth-serving-cert\") pod \"console-7f6cd4fd5c-csqmm\" (UID: \"fe433fb2-8705-4c66-bc23-4cae16489508\") " pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:08.195428 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.195375    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe433fb2-8705-4c66-bc23-4cae16489508-service-ca\") pod \"console-7f6cd4fd5c-csqmm\" (UID: \"fe433fb2-8705-4c66-bc23-4cae16489508\") " pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:08.195512 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.195494    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe433fb2-8705-4c66-bc23-4cae16489508-trusted-ca-bundle\") pod \"console-7f6cd4fd5c-csqmm\" (UID: \"fe433fb2-8705-4c66-bc23-4cae16489508\") " pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:08.197014 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.196993    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe433fb2-8705-4c66-bc23-4cae16489508-console-serving-cert\") pod \"console-7f6cd4fd5c-csqmm\" (UID: \"fe433fb2-8705-4c66-bc23-4cae16489508\") " pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:08.197115 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.197065    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe433fb2-8705-4c66-bc23-4cae16489508-console-oauth-config\") pod \"console-7f6cd4fd5c-csqmm\" (UID: \"fe433fb2-8705-4c66-bc23-4cae16489508\") " pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:08.211773 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.211748    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr8sx\" (UniqueName: \"kubernetes.io/projected/fe433fb2-8705-4c66-bc23-4cae16489508-kube-api-access-kr8sx\") pod \"console-7f6cd4fd5c-csqmm\" (UID: \"fe433fb2-8705-4c66-bc23-4cae16489508\") " pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:08.253751 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.253722    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:08.403814 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:08.403776    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f6cd4fd5c-csqmm"]
Jul 15 17:08:08.407274 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:08:08.407243    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe433fb2_8705_4c66_bc23_4cae16489508.slice/crio-591a580a6563d6654681c54d89a7f015d22716a0906e88ffb6ba1124ffadbce4 WatchSource:0}: Error finding container 591a580a6563d6654681c54d89a7f015d22716a0906e88ffb6ba1124ffadbce4: Status 404 returned error can't find the container with id 591a580a6563d6654681c54d89a7f015d22716a0906e88ffb6ba1124ffadbce4
Jul 15 17:08:09.030150 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:09.030114    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f6cd4fd5c-csqmm" event={"ID":"fe433fb2-8705-4c66-bc23-4cae16489508","Type":"ContainerStarted","Data":"5a5831c93644f77205f8f6e4cccdbd86019eecff7d210f54db45ab93c80a0b5e"}
Jul 15 17:08:09.030589 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:09.030171    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f6cd4fd5c-csqmm" event={"ID":"fe433fb2-8705-4c66-bc23-4cae16489508","Type":"ContainerStarted","Data":"591a580a6563d6654681c54d89a7f015d22716a0906e88ffb6ba1124ffadbce4"}
Jul 15 17:08:09.049844 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:09.049782    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f6cd4fd5c-csqmm" podStartSLOduration=2.049765647 podStartE2EDuration="2.049765647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:08:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:08:09.048060446 +0000 UTC m=+705.065473566" watchObservedRunningTime="2026-07-15 17:08:09.049765647 +0000 UTC m=+705.067178803"
Jul 15 17:08:10.845483 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:10.845443    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-z6dxh"]
Jul 15 17:08:10.849320 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:10.849300    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z6dxh"
Jul 15 17:08:10.852040 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:10.852016    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-c856k\""
Jul 15 17:08:10.857868 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:10.857702    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-z6dxh"]
Jul 15 17:08:11.019496 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:11.019474    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh8h6\" (UniqueName: \"kubernetes.io/projected/54f5f063-919a-4b65-b250-feb767ff295d-kube-api-access-kh8h6\") pod \"authorino-operator-75cc8d5cb7-z6dxh\" (UID: \"54f5f063-919a-4b65-b250-feb767ff295d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-z6dxh"
Jul 15 17:08:11.044037 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:11.044003    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj" event={"ID":"5c225c69-7a29-4c0e-9f4c-fcc69030dbff","Type":"ContainerStarted","Data":"80cef66ce35d7d68dea7bec7753989c4d5ebd068b63d8e9a0c9fafa6d09a2aef"}
Jul 15 17:08:11.044174 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:11.044142    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:11.045885 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:11.045861    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph" event={"ID":"7b92d62d-3a24-4262-bab3-e8b799adbbfe","Type":"ContainerStarted","Data":"7f235c89a3cd79cb53de225b20e6763c16a231dbdf6efc7e40bc4fcd9289a391"}
Jul 15 17:08:11.046009 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:11.045995    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"
Jul 15 17:08:11.084201 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:11.084150    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph" podStartSLOduration=0.91721753 podStartE2EDuration="4.084135671s" totalImagesPullingTime="3.166918141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:08:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:08:11.083599932 +0000 UTC m=+707.101013043" watchObservedRunningTime="2026-07-15 17:08:11.084135671 +0000 UTC m=+707.101548768"
Jul 15 17:08:11.085080 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:11.085047    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj" podStartSLOduration=1.923072154 podStartE2EDuration="9.085035769s" totalImagesPullingTime="7.161963615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:08:11.062338017 +0000 UTC m=+707.079751125" watchObservedRunningTime="2026-07-15 17:08:11.085035769 +0000 UTC m=+707.102448869"
Jul 15 17:08:11.120314 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:11.120293    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh8h6\" (UniqueName: \"kubernetes.io/projected/54f5f063-919a-4b65-b250-feb767ff295d-kube-api-access-kh8h6\") pod \"authorino-operator-75cc8d5cb7-z6dxh\" (UID: \"54f5f063-919a-4b65-b250-feb767ff295d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-z6dxh"
Jul 15 17:08:11.128348 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:11.128332    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh8h6\" (UniqueName: \"kubernetes.io/projected/54f5f063-919a-4b65-b250-feb767ff295d-kube-api-access-kh8h6\") pod \"authorino-operator-75cc8d5cb7-z6dxh\" (UID: \"54f5f063-919a-4b65-b250-feb767ff295d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-z6dxh"
Jul 15 17:08:11.162509 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:11.162479    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z6dxh"
Jul 15 17:08:11.297393 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:11.297356    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-z6dxh"]
Jul 15 17:08:11.299626 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:08:11.299596    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54f5f063_919a_4b65_b250_feb767ff295d.slice/crio-ee2f79c081a4b0e4e1f43a3ac45990708d070de1b5e2ba7d12acbbd00176948d WatchSource:0}: Error finding container ee2f79c081a4b0e4e1f43a3ac45990708d070de1b5e2ba7d12acbbd00176948d: Status 404 returned error can't find the container with id ee2f79c081a4b0e4e1f43a3ac45990708d070de1b5e2ba7d12acbbd00176948d
Jul 15 17:08:12.053308 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:12.053254    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z6dxh" event={"ID":"54f5f063-919a-4b65-b250-feb767ff295d","Type":"ContainerStarted","Data":"ee2f79c081a4b0e4e1f43a3ac45990708d070de1b5e2ba7d12acbbd00176948d"}
Jul 15 17:08:13.084573 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:13.084522    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-nmtff"]
Jul 15 17:08:13.095111 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:13.095086    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-nmtff"]
Jul 15 17:08:13.095252 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:13.095186    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nmtff"
Jul 15 17:08:13.098102 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:13.098013    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fx4k8\""
Jul 15 17:08:13.098102 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:13.098064    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 15 17:08:13.098315 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:13.098070    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 15 17:08:13.240836 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:13.240802    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6facd58-286a-47fc-9226-5ccdb6d8113e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nmtff\" (UID: \"b6facd58-286a-47fc-9226-5ccdb6d8113e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nmtff"
Jul 15 17:08:13.240836 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:13.240839    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2j8t\" (UniqueName: \"kubernetes.io/projected/b6facd58-286a-47fc-9226-5ccdb6d8113e-kube-api-access-v2j8t\") pod \"kuadrant-console-plugin-66b5774985-nmtff\" (UID: \"b6facd58-286a-47fc-9226-5ccdb6d8113e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nmtff"
Jul 15 17:08:13.241039 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:13.240939    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b6facd58-286a-47fc-9226-5ccdb6d8113e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-nmtff\" (UID: \"b6facd58-286a-47fc-9226-5ccdb6d8113e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nmtff"
Jul 15 17:08:13.341869 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:13.341772    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6facd58-286a-47fc-9226-5ccdb6d8113e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nmtff\" (UID: \"b6facd58-286a-47fc-9226-5ccdb6d8113e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nmtff"
Jul 15 17:08:13.341869 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:13.341818    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2j8t\" (UniqueName: \"kubernetes.io/projected/b6facd58-286a-47fc-9226-5ccdb6d8113e-kube-api-access-v2j8t\") pod \"kuadrant-console-plugin-66b5774985-nmtff\" (UID: \"b6facd58-286a-47fc-9226-5ccdb6d8113e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nmtff"
Jul 15 17:08:13.342090 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:13.341870    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b6facd58-286a-47fc-9226-5ccdb6d8113e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-nmtff\" (UID: \"b6facd58-286a-47fc-9226-5ccdb6d8113e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nmtff"
Jul 15 17:08:13.342616 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:13.342592    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b6facd58-286a-47fc-9226-5ccdb6d8113e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-nmtff\" (UID: \"b6facd58-286a-47fc-9226-5ccdb6d8113e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nmtff"
Jul 15 17:08:13.344475 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:13.344451    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6facd58-286a-47fc-9226-5ccdb6d8113e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nmtff\" (UID: \"b6facd58-286a-47fc-9226-5ccdb6d8113e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nmtff"
Jul 15 17:08:13.350859 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:13.350825    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2j8t\" (UniqueName: \"kubernetes.io/projected/b6facd58-286a-47fc-9226-5ccdb6d8113e-kube-api-access-v2j8t\") pod \"kuadrant-console-plugin-66b5774985-nmtff\" (UID: \"b6facd58-286a-47fc-9226-5ccdb6d8113e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nmtff"
Jul 15 17:08:13.410344 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:13.410300    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nmtff"
Jul 15 17:08:13.658716 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:13.658695    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-nmtff"]
Jul 15 17:08:13.660111 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:08:13.660090    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6facd58_286a_47fc_9226_5ccdb6d8113e.slice/crio-3f9502c03687cc5dfb4fc84115be00c5758721bc82c8d1999359d6c032c02994 WatchSource:0}: Error finding container 3f9502c03687cc5dfb4fc84115be00c5758721bc82c8d1999359d6c032c02994: Status 404 returned error can't find the container with id 3f9502c03687cc5dfb4fc84115be00c5758721bc82c8d1999359d6c032c02994
Jul 15 17:08:14.068449 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:14.068411    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z6dxh" event={"ID":"54f5f063-919a-4b65-b250-feb767ff295d","Type":"ContainerStarted","Data":"08d17de7cfb3df7b69bcba6319bfb853f8669d4c4e37c9529799d02042e557c1"}
Jul 15 17:08:14.068650 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:14.068514    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z6dxh"
Jul 15 17:08:14.069580 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:14.069546    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nmtff" event={"ID":"b6facd58-286a-47fc-9226-5ccdb6d8113e","Type":"ContainerStarted","Data":"3f9502c03687cc5dfb4fc84115be00c5758721bc82c8d1999359d6c032c02994"}
Jul 15 17:08:14.086948 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:14.086902    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z6dxh" podStartSLOduration=1.805192924 podStartE2EDuration="4.086889569s" totalImagesPullingTime="2.281696645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:08:14.085693371 +0000 UTC m=+710.103106485" watchObservedRunningTime="2026-07-15 17:08:14.086889569 +0000 UTC m=+710.104302665"
Jul 15 17:08:18.026580 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:18.026498    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f7dfk"
Jul 15 17:08:18.254974 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:18.254937    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:18.255139 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:18.255009    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:18.261536 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:18.261505    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:19.100408 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:19.100376    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f6cd4fd5c-csqmm"
Jul 15 17:08:19.160594 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:19.160189    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7ffd9bcfc9-zz9cf"]
Jul 15 17:08:22.056042 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:22.056015    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:22.056444 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:22.056094    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"
Jul 15 17:08:23.814774 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:23.814739    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"]
Jul 15 17:08:23.815194 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:23.815013    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj" podUID="5c225c69-7a29-4c0e-9f4c-fcc69030dbff" containerName="manager" containerID="cri-o://80cef66ce35d7d68dea7bec7753989c4d5ebd068b63d8e9a0c9fafa6d09a2aef" gracePeriod=2
Jul 15 17:08:23.817379 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:23.817340    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-js9pj\" is forbidden: User \"system:node:ip-10-0-142-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-51.ec2.internal' and this object" podUID="5c225c69-7a29-4c0e-9f4c-fcc69030dbff" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:23.819177 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:23.819152    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"]
Jul 15 17:08:23.826537 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:23.826509    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"]
Jul 15 17:08:23.828371 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:23.826752    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph" podUID="7b92d62d-3a24-4262-bab3-e8b799adbbfe" containerName="manager" containerID="cri-o://7f235c89a3cd79cb53de225b20e6763c16a231dbdf6efc7e40bc4fcd9289a391" gracePeriod=2
Jul 15 17:08:23.844300 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:23.844035    2575 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"7b92d62d-3a24-4262-bab3-e8b799adbbfe\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-6d2ph\": pods \"limitador-operator-controller-manager-859d7b78bd-6d2ph\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"
Jul 15 17:08:23.846834 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:23.846805    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-js9pj\" is forbidden: User \"system:node:ip-10-0-142-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-51.ec2.internal' and this object" podUID="5c225c69-7a29-4c0e-9f4c-fcc69030dbff" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:23.847836 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:23.847817    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"]
Jul 15 17:08:23.849373 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:23.849329    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv"]
Jul 15 17:08:23.849968 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:23.849949    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c225c69-7a29-4c0e-9f4c-fcc69030dbff" containerName="manager"
Jul 15 17:08:23.850057 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:23.849970    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c225c69-7a29-4c0e-9f4c-fcc69030dbff" containerName="manager"
Jul 15 17:08:23.850057 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:23.849982    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b92d62d-3a24-4262-bab3-e8b799adbbfe" containerName="manager"
Jul 15 17:08:23.850057 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:23.849989    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b92d62d-3a24-4262-bab3-e8b799adbbfe" containerName="manager"
Jul 15 17:08:23.850201 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:23.850065    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c225c69-7a29-4c0e-9f4c-fcc69030dbff" containerName="manager"
Jul 15 17:08:23.850201 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:23.850078    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b92d62d-3a24-4262-bab3-e8b799adbbfe" containerName="manager"
Jul 15 17:08:23.853253 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:23.853221    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv"
Jul 15 17:08:23.854596 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:23.854572    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9p25"]
Jul 15 17:08:23.858105 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:23.858085    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9p25"
Jul 15 17:08:23.862546 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:23.862497    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv"]
Jul 15 17:08:23.868339 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:23.868316    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9p25"]
Jul 15 17:08:23.880278 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:23.880244    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6d2ph\" is forbidden: User \"system:node:ip-10-0-142-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-51.ec2.internal' and this object" podUID="7b92d62d-3a24-4262-bab3-e8b799adbbfe" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"
Jul 15 17:08:23.882208 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:23.882184    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-js9pj\" is forbidden: User \"system:node:ip-10-0-142-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-51.ec2.internal' and this object" podUID="5c225c69-7a29-4c0e-9f4c-fcc69030dbff" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:23.884109 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:23.884080    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-js9pj\" is forbidden: User \"system:node:ip-10-0-142-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-51.ec2.internal' and this object" podUID="5c225c69-7a29-4c0e-9f4c-fcc69030dbff" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:23.901928 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:23.901904    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6d2ph\" is forbidden: User \"system:node:ip-10-0-142-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-51.ec2.internal' and this object" podUID="7b92d62d-3a24-4262-bab3-e8b799adbbfe" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"
Jul 15 17:08:23.928004 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:23.927983    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxbz2\" (UniqueName: \"kubernetes.io/projected/7d57e791-c22c-424f-9612-1960f235de30-kube-api-access-mxbz2\") pod \"limitador-operator-controller-manager-859d7b78bd-w9p25\" (UID: \"7d57e791-c22c-424f-9612-1960f235de30\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9p25"
Jul 15 17:08:23.928107 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:23.928025    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d2b50e4-b33f-49e1-9129-9d7d0158dc7f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mcvwv\" (UID: \"9d2b50e4-b33f-49e1-9129-9d7d0158dc7f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv"
Jul 15 17:08:23.928107 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:23.928050    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6dt4\" (UniqueName: \"kubernetes.io/projected/9d2b50e4-b33f-49e1-9129-9d7d0158dc7f-kube-api-access-d6dt4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mcvwv\" (UID: \"9d2b50e4-b33f-49e1-9129-9d7d0158dc7f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv"
Jul 15 17:08:24.029523 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.029491    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxbz2\" (UniqueName: \"kubernetes.io/projected/7d57e791-c22c-424f-9612-1960f235de30-kube-api-access-mxbz2\") pod \"limitador-operator-controller-manager-859d7b78bd-w9p25\" (UID: \"7d57e791-c22c-424f-9612-1960f235de30\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9p25"
Jul 15 17:08:24.029677 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.029567    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d2b50e4-b33f-49e1-9129-9d7d0158dc7f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mcvwv\" (UID: \"9d2b50e4-b33f-49e1-9129-9d7d0158dc7f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv"
Jul 15 17:08:24.029677 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.029605    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6dt4\" (UniqueName: \"kubernetes.io/projected/9d2b50e4-b33f-49e1-9129-9d7d0158dc7f-kube-api-access-d6dt4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mcvwv\" (UID: \"9d2b50e4-b33f-49e1-9129-9d7d0158dc7f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv"
Jul 15 17:08:24.030128 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.030101    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d2b50e4-b33f-49e1-9129-9d7d0158dc7f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mcvwv\" (UID: \"9d2b50e4-b33f-49e1-9129-9d7d0158dc7f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv"
Jul 15 17:08:24.039934 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.039884    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxbz2\" (UniqueName: \"kubernetes.io/projected/7d57e791-c22c-424f-9612-1960f235de30-kube-api-access-mxbz2\") pod \"limitador-operator-controller-manager-859d7b78bd-w9p25\" (UID: \"7d57e791-c22c-424f-9612-1960f235de30\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9p25"
Jul 15 17:08:24.039934 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.039922    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6dt4\" (UniqueName: \"kubernetes.io/projected/9d2b50e4-b33f-49e1-9129-9d7d0158dc7f-kube-api-access-d6dt4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mcvwv\" (UID: \"9d2b50e4-b33f-49e1-9129-9d7d0158dc7f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv"
Jul 15 17:08:24.066810 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.066757    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:24.068930 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:24.068902    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-js9pj\" is forbidden: User \"system:node:ip-10-0-142-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-51.ec2.internal' and this object" podUID="5c225c69-7a29-4c0e-9f4c-fcc69030dbff" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:24.070966 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:24.070935    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6d2ph\" is forbidden: User \"system:node:ip-10-0-142-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-51.ec2.internal' and this object" podUID="7b92d62d-3a24-4262-bab3-e8b799adbbfe" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"
Jul 15 17:08:24.093067 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.093041    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"
Jul 15 17:08:24.095338 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:24.095310    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6d2ph\" is forbidden: User \"system:node:ip-10-0-142-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-51.ec2.internal' and this object" podUID="7b92d62d-3a24-4262-bab3-e8b799adbbfe" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"
Jul 15 17:08:24.097294 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:24.097269    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-js9pj\" is forbidden: User \"system:node:ip-10-0-142-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-51.ec2.internal' and this object" podUID="5c225c69-7a29-4c0e-9f4c-fcc69030dbff" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:24.121979 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.121953    2575 generic.go:356] "Generic (PLEG): container finished" podID="5c225c69-7a29-4c0e-9f4c-fcc69030dbff" containerID="80cef66ce35d7d68dea7bec7753989c4d5ebd068b63d8e9a0c9fafa6d09a2aef" exitCode=0
Jul 15 17:08:24.122095 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.122003    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:24.122095 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.122069    2575 scope.go:117] "RemoveContainer" containerID="80cef66ce35d7d68dea7bec7753989c4d5ebd068b63d8e9a0c9fafa6d09a2aef"
Jul 15 17:08:24.123292 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.123272    2575 generic.go:356] "Generic (PLEG): container finished" podID="7b92d62d-3a24-4262-bab3-e8b799adbbfe" containerID="7f235c89a3cd79cb53de225b20e6763c16a231dbdf6efc7e40bc4fcd9289a391" exitCode=0
Jul 15 17:08:24.123365 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.123308    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"
Jul 15 17:08:24.124492 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:24.124470    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6d2ph\" is forbidden: User \"system:node:ip-10-0-142-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-51.ec2.internal' and this object" podUID="7b92d62d-3a24-4262-bab3-e8b799adbbfe" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"
Jul 15 17:08:24.126318 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:24.126294    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-js9pj\" is forbidden: User \"system:node:ip-10-0-142-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-51.ec2.internal' and this object" podUID="5c225c69-7a29-4c0e-9f4c-fcc69030dbff" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:24.128178 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:24.128158    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6d2ph\" is forbidden: User \"system:node:ip-10-0-142-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-51.ec2.internal' and this object" podUID="7b92d62d-3a24-4262-bab3-e8b799adbbfe" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"
Jul 15 17:08:24.129970 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.129953    2575 scope.go:117] "RemoveContainer" containerID="80cef66ce35d7d68dea7bec7753989c4d5ebd068b63d8e9a0c9fafa6d09a2aef"
Jul 15 17:08:24.130038 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:24.130005    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-js9pj\" is forbidden: User \"system:node:ip-10-0-142-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-51.ec2.internal' and this object" podUID="5c225c69-7a29-4c0e-9f4c-fcc69030dbff" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:24.130146 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.130129    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f99x5\" (UniqueName: \"kubernetes.io/projected/5c225c69-7a29-4c0e-9f4c-fcc69030dbff-kube-api-access-f99x5\") pod \"5c225c69-7a29-4c0e-9f4c-fcc69030dbff\" (UID: \"5c225c69-7a29-4c0e-9f4c-fcc69030dbff\") "
Jul 15 17:08:24.130185 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.130166    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c225c69-7a29-4c0e-9f4c-fcc69030dbff-extensions-socket-volume\") pod \"5c225c69-7a29-4c0e-9f4c-fcc69030dbff\" (UID: \"5c225c69-7a29-4c0e-9f4c-fcc69030dbff\") "
Jul 15 17:08:24.130224 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.130213    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7gkpx\" (UniqueName: \"kubernetes.io/projected/7b92d62d-3a24-4262-bab3-e8b799adbbfe-kube-api-access-7gkpx\") pod \"7b92d62d-3a24-4262-bab3-e8b799adbbfe\" (UID: \"7b92d62d-3a24-4262-bab3-e8b799adbbfe\") "
Jul 15 17:08:24.130342 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:24.130323    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80cef66ce35d7d68dea7bec7753989c4d5ebd068b63d8e9a0c9fafa6d09a2aef\": container with ID starting with 80cef66ce35d7d68dea7bec7753989c4d5ebd068b63d8e9a0c9fafa6d09a2aef not found: ID does not exist" containerID="80cef66ce35d7d68dea7bec7753989c4d5ebd068b63d8e9a0c9fafa6d09a2aef"
Jul 15 17:08:24.130403 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.130353    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80cef66ce35d7d68dea7bec7753989c4d5ebd068b63d8e9a0c9fafa6d09a2aef"} err="failed to get container status \"80cef66ce35d7d68dea7bec7753989c4d5ebd068b63d8e9a0c9fafa6d09a2aef\": rpc error: code = NotFound desc = could not find container \"80cef66ce35d7d68dea7bec7753989c4d5ebd068b63d8e9a0c9fafa6d09a2aef\": container with ID starting with 80cef66ce35d7d68dea7bec7753989c4d5ebd068b63d8e9a0c9fafa6d09a2aef not found: ID does not exist"
Jul 15 17:08:24.130403 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.130377    2575 scope.go:117] "RemoveContainer" containerID="7f235c89a3cd79cb53de225b20e6763c16a231dbdf6efc7e40bc4fcd9289a391"
Jul 15 17:08:24.130763 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.130735    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c225c69-7a29-4c0e-9f4c-fcc69030dbff-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5c225c69-7a29-4c0e-9f4c-fcc69030dbff" (UID: "5c225c69-7a29-4c0e-9f4c-fcc69030dbff"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:08:24.132116 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.132095    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c225c69-7a29-4c0e-9f4c-fcc69030dbff-kube-api-access-f99x5" (OuterVolumeSpecName: "kube-api-access-f99x5") pod "5c225c69-7a29-4c0e-9f4c-fcc69030dbff" (UID: "5c225c69-7a29-4c0e-9f4c-fcc69030dbff"). InnerVolumeSpecName "kube-api-access-f99x5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:08:24.132318 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.132300    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b92d62d-3a24-4262-bab3-e8b799adbbfe-kube-api-access-7gkpx" (OuterVolumeSpecName: "kube-api-access-7gkpx") pod "7b92d62d-3a24-4262-bab3-e8b799adbbfe" (UID: "7b92d62d-3a24-4262-bab3-e8b799adbbfe"). InnerVolumeSpecName "kube-api-access-7gkpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:08:24.137899 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.137884    2575 scope.go:117] "RemoveContainer" containerID="7f235c89a3cd79cb53de225b20e6763c16a231dbdf6efc7e40bc4fcd9289a391"
Jul 15 17:08:24.138140 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:24.138122    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f235c89a3cd79cb53de225b20e6763c16a231dbdf6efc7e40bc4fcd9289a391\": container with ID starting with 7f235c89a3cd79cb53de225b20e6763c16a231dbdf6efc7e40bc4fcd9289a391 not found: ID does not exist" containerID="7f235c89a3cd79cb53de225b20e6763c16a231dbdf6efc7e40bc4fcd9289a391"
Jul 15 17:08:24.138182 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.138148    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f235c89a3cd79cb53de225b20e6763c16a231dbdf6efc7e40bc4fcd9289a391"} err="failed to get container status \"7f235c89a3cd79cb53de225b20e6763c16a231dbdf6efc7e40bc4fcd9289a391\": rpc error: code = NotFound desc = could not find container \"7f235c89a3cd79cb53de225b20e6763c16a231dbdf6efc7e40bc4fcd9289a391\": container with ID starting with 7f235c89a3cd79cb53de225b20e6763c16a231dbdf6efc7e40bc4fcd9289a391 not found: ID does not exist"
Jul 15 17:08:24.231757 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.231724    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f99x5\" (UniqueName: \"kubernetes.io/projected/5c225c69-7a29-4c0e-9f4c-fcc69030dbff-kube-api-access-f99x5\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:08:24.231757 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.231754    2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c225c69-7a29-4c0e-9f4c-fcc69030dbff-extensions-socket-volume\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:08:24.231981 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.231773    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gkpx\" (UniqueName: \"kubernetes.io/projected/7b92d62d-3a24-4262-bab3-e8b799adbbfe-kube-api-access-7gkpx\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:08:24.253738 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.253707    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv"
Jul 15 17:08:24.262415 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.262394    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9p25"
Jul 15 17:08:24.438534 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:24.438496    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6d2ph\" is forbidden: User \"system:node:ip-10-0-142-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-51.ec2.internal' and this object" podUID="7b92d62d-3a24-4262-bab3-e8b799adbbfe" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"
Jul 15 17:08:24.440967 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:24.440935    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-js9pj\" is forbidden: User \"system:node:ip-10-0-142-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-51.ec2.internal' and this object" podUID="5c225c69-7a29-4c0e-9f4c-fcc69030dbff" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:24.443173 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:24.443146    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6d2ph\" is forbidden: User \"system:node:ip-10-0-142-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-51.ec2.internal' and this object" podUID="7b92d62d-3a24-4262-bab3-e8b799adbbfe" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"
Jul 15 17:08:24.445650 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:24.445621    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-js9pj\" is forbidden: User \"system:node:ip-10-0-142-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-51.ec2.internal' and this object" podUID="5c225c69-7a29-4c0e-9f4c-fcc69030dbff" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:24.527545 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:24.527509    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-js9pj\" is forbidden: User \"system:node:ip-10-0-142-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-51.ec2.internal' and this object" podUID="5c225c69-7a29-4c0e-9f4c-fcc69030dbff" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-js9pj"
Jul 15 17:08:24.527732 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.527631    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c225c69-7a29-4c0e-9f4c-fcc69030dbff" path="/var/lib/kubelet/pods/5c225c69-7a29-4c0e-9f4c-fcc69030dbff/volumes"
Jul 15 17:08:24.528069 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:24.528049    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b92d62d-3a24-4262-bab3-e8b799adbbfe" path="/var/lib/kubelet/pods/7b92d62d-3a24-4262-bab3-e8b799adbbfe/volumes"
Jul 15 17:08:24.532156 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:24.532123    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6d2ph\" is forbidden: User \"system:node:ip-10-0-142-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-51.ec2.internal' and this object" podUID="7b92d62d-3a24-4262-bab3-e8b799adbbfe" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6d2ph"
Jul 15 17:08:25.077931 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:25.077900    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z6dxh"
Jul 15 17:08:38.685492 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:38.685462    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv"]
Jul 15 17:08:38.687058 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:08:38.687024    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d2b50e4_b33f_49e1_9129_9d7d0158dc7f.slice/crio-53bf3db54c62aa4941cf445ff94ee794d2a8bfd29b8df22b0fe9e6acf99936f7 WatchSource:0}: Error finding container 53bf3db54c62aa4941cf445ff94ee794d2a8bfd29b8df22b0fe9e6acf99936f7: Status 404 returned error can't find the container with id 53bf3db54c62aa4941cf445ff94ee794d2a8bfd29b8df22b0fe9e6acf99936f7
Jul 15 17:08:38.708859 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:38.708834    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9p25"]
Jul 15 17:08:38.711082 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:08:38.711059    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d57e791_c22c_424f_9612_1960f235de30.slice/crio-de470e7efea4a06ea466c7a2622d7e307b653103c25e6ff44b64b648a2510b92 WatchSource:0}: Error finding container de470e7efea4a06ea466c7a2622d7e307b653103c25e6ff44b64b648a2510b92: Status 404 returned error can't find the container with id de470e7efea4a06ea466c7a2622d7e307b653103c25e6ff44b64b648a2510b92
Jul 15 17:08:39.202997 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:39.202878    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nmtff" event={"ID":"b6facd58-286a-47fc-9226-5ccdb6d8113e","Type":"ContainerStarted","Data":"31c295a42bf6332e1d0ed226d41a3f19d592ac1735d465a55d8aeb48810eecd1"}
Jul 15 17:08:39.204527 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:39.204502    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9p25" event={"ID":"7d57e791-c22c-424f-9612-1960f235de30","Type":"ContainerStarted","Data":"11b71da7e65bf45475f15bbc8f1c7bd24d9271c9f5c7fac50ff65b989918ef31"}
Jul 15 17:08:39.204672 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:39.204544    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9p25" event={"ID":"7d57e791-c22c-424f-9612-1960f235de30","Type":"ContainerStarted","Data":"de470e7efea4a06ea466c7a2622d7e307b653103c25e6ff44b64b648a2510b92"}
Jul 15 17:08:39.204672 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:39.204589    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9p25"
Jul 15 17:08:39.206022 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:39.205997    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv" event={"ID":"9d2b50e4-b33f-49e1-9129-9d7d0158dc7f","Type":"ContainerStarted","Data":"89bfbfbb71cbed3006654481f3276d2cd589b9b9c7855a9f161b010b7fbd2efe"}
Jul 15 17:08:39.206118 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:39.206044    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv" event={"ID":"9d2b50e4-b33f-49e1-9129-9d7d0158dc7f","Type":"ContainerStarted","Data":"53bf3db54c62aa4941cf445ff94ee794d2a8bfd29b8df22b0fe9e6acf99936f7"}
Jul 15 17:08:39.206118 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:39.206072    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv"
Jul 15 17:08:39.218197 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:39.218094    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nmtff" podStartSLOduration=0.896603887 podStartE2EDuration="26.218076296s" totalImagesPullingTime="25.321472409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:08:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:08:39.217790022 +0000 UTC m=+735.235203140" watchObservedRunningTime="2026-07-15 17:08:39.218076296 +0000 UTC m=+735.235489393"
Jul 15 17:08:39.237430 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:39.237380    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv" podStartSLOduration=16.237365814 podStartE2EDuration="16.237365814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:08:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:08:39.235781404 +0000 UTC m=+735.253194517" watchObservedRunningTime="2026-07-15 17:08:39.237365814 +0000 UTC m=+735.254778909"
Jul 15 17:08:39.252435 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:39.252212    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9p25" podStartSLOduration=16.252194236 podStartE2EDuration="16.252194236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:08:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:08:39.250842339 +0000 UTC m=+735.268255460" watchObservedRunningTime="2026-07-15 17:08:39.252194236 +0000 UTC m=+735.269607334"
Jul 15 17:08:41.997579 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:41.997528    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv"]
Jul 15 17:08:41.998025 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:41.997727    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv" podUID="9d2b50e4-b33f-49e1-9129-9d7d0158dc7f" containerName="manager" containerID="cri-o://89bfbfbb71cbed3006654481f3276d2cd589b9b9c7855a9f161b010b7fbd2efe" gracePeriod=10
Jul 15 17:08:42.221048 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:42.221018    2575 generic.go:356] "Generic (PLEG): container finished" podID="9d2b50e4-b33f-49e1-9129-9d7d0158dc7f" containerID="89bfbfbb71cbed3006654481f3276d2cd589b9b9c7855a9f161b010b7fbd2efe" exitCode=0
Jul 15 17:08:42.221218 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:42.221092    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv" event={"ID":"9d2b50e4-b33f-49e1-9129-9d7d0158dc7f","Type":"ContainerDied","Data":"89bfbfbb71cbed3006654481f3276d2cd589b9b9c7855a9f161b010b7fbd2efe"}
Jul 15 17:08:42.785540 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:42.785513    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv"
Jul 15 17:08:42.904310 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:42.904233    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d2b50e4-b33f-49e1-9129-9d7d0158dc7f-extensions-socket-volume\") pod \"9d2b50e4-b33f-49e1-9129-9d7d0158dc7f\" (UID: \"9d2b50e4-b33f-49e1-9129-9d7d0158dc7f\") "
Jul 15 17:08:42.904310 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:42.904297    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6dt4\" (UniqueName: \"kubernetes.io/projected/9d2b50e4-b33f-49e1-9129-9d7d0158dc7f-kube-api-access-d6dt4\") pod \"9d2b50e4-b33f-49e1-9129-9d7d0158dc7f\" (UID: \"9d2b50e4-b33f-49e1-9129-9d7d0158dc7f\") "
Jul 15 17:08:42.904676 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:42.904652    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d2b50e4-b33f-49e1-9129-9d7d0158dc7f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9d2b50e4-b33f-49e1-9129-9d7d0158dc7f" (UID: "9d2b50e4-b33f-49e1-9129-9d7d0158dc7f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:08:42.906317 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:42.906296    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d2b50e4-b33f-49e1-9129-9d7d0158dc7f-kube-api-access-d6dt4" (OuterVolumeSpecName: "kube-api-access-d6dt4") pod "9d2b50e4-b33f-49e1-9129-9d7d0158dc7f" (UID: "9d2b50e4-b33f-49e1-9129-9d7d0158dc7f"). InnerVolumeSpecName "kube-api-access-d6dt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:08:43.005425 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:43.005391    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6dt4\" (UniqueName: \"kubernetes.io/projected/9d2b50e4-b33f-49e1-9129-9d7d0158dc7f-kube-api-access-d6dt4\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:08:43.005425 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:43.005418    2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d2b50e4-b33f-49e1-9129-9d7d0158dc7f-extensions-socket-volume\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:08:43.226401 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:43.226330    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv"
Jul 15 17:08:43.226401 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:43.226368    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv" event={"ID":"9d2b50e4-b33f-49e1-9129-9d7d0158dc7f","Type":"ContainerDied","Data":"53bf3db54c62aa4941cf445ff94ee794d2a8bfd29b8df22b0fe9e6acf99936f7"}
Jul 15 17:08:43.226625 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:43.226429    2575 scope.go:117] "RemoveContainer" containerID="89bfbfbb71cbed3006654481f3276d2cd589b9b9c7855a9f161b010b7fbd2efe"
Jul 15 17:08:43.257716 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:43.257689    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv"]
Jul 15 17:08:43.259924 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:43.259902    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mcvwv"]
Jul 15 17:08:44.191389 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:44.191344    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7ffd9bcfc9-zz9cf" podUID="62d4bd49-f0c6-4bd5-bebf-1e68b96d780c" containerName="console" containerID="cri-o://7bc9b4f48a8cd127ac80b346e0785d7a35ff6db039b7e29d846ea3c7b91de8e7" gracePeriod=15
Jul 15 17:08:44.524911 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:44.524836    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d2b50e4-b33f-49e1-9129-9d7d0158dc7f" path="/var/lib/kubelet/pods/9d2b50e4-b33f-49e1-9129-9d7d0158dc7f/volumes"
Jul 15 17:08:44.962805 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:44.962778    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:08:45.122439 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.122413    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-trusted-ca-bundle\") pod \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") "
Jul 15 17:08:45.122607 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.122465    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-service-ca\") pod \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") "
Jul 15 17:08:45.122607 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.122499    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kl84g\" (UniqueName: \"kubernetes.io/projected/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-kube-api-access-kl84g\") pod \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") "
Jul 15 17:08:45.122607 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.122543    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-console-oauth-config\") pod \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") "
Jul 15 17:08:45.122607 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.122588    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-console-config\") pod \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") "
Jul 15 17:08:45.122828 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.122669    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-console-serving-cert\") pod \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") "
Jul 15 17:08:45.122828 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.122702    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-oauth-serving-cert\") pod \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\" (UID: \"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c\") "
Jul 15 17:08:45.122932 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.122848    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "62d4bd49-f0c6-4bd5-bebf-1e68b96d780c" (UID: "62d4bd49-f0c6-4bd5-bebf-1e68b96d780c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:08:45.122984 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.122955    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-service-ca" (OuterVolumeSpecName: "service-ca") pod "62d4bd49-f0c6-4bd5-bebf-1e68b96d780c" (UID: "62d4bd49-f0c6-4bd5-bebf-1e68b96d780c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:08:45.122984 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.122965    2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-trusted-ca-bundle\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:08:45.123181 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.123155    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "62d4bd49-f0c6-4bd5-bebf-1e68b96d780c" (UID: "62d4bd49-f0c6-4bd5-bebf-1e68b96d780c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:08:45.123314 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.123176    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-console-config" (OuterVolumeSpecName: "console-config") pod "62d4bd49-f0c6-4bd5-bebf-1e68b96d780c" (UID: "62d4bd49-f0c6-4bd5-bebf-1e68b96d780c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:08:45.124883 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.124853    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "62d4bd49-f0c6-4bd5-bebf-1e68b96d780c" (UID: "62d4bd49-f0c6-4bd5-bebf-1e68b96d780c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:08:45.124971 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.124944    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-kube-api-access-kl84g" (OuterVolumeSpecName: "kube-api-access-kl84g") pod "62d4bd49-f0c6-4bd5-bebf-1e68b96d780c" (UID: "62d4bd49-f0c6-4bd5-bebf-1e68b96d780c"). InnerVolumeSpecName "kube-api-access-kl84g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:08:45.125177 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.125153    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "62d4bd49-f0c6-4bd5-bebf-1e68b96d780c" (UID: "62d4bd49-f0c6-4bd5-bebf-1e68b96d780c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:08:45.224100 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.224058    2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-console-serving-cert\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:08:45.224100 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.224095    2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-oauth-serving-cert\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:08:45.224100 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.224108    2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-service-ca\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:08:45.224540 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.224120    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kl84g\" (UniqueName: \"kubernetes.io/projected/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-kube-api-access-kl84g\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:08:45.224540 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.224132    2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-console-oauth-config\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:08:45.224540 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.224142    2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c-console-config\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:08:45.238327 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.238293    2575 generic.go:356] "Generic (PLEG): container finished" podID="62d4bd49-f0c6-4bd5-bebf-1e68b96d780c" containerID="7bc9b4f48a8cd127ac80b346e0785d7a35ff6db039b7e29d846ea3c7b91de8e7" exitCode=0
Jul 15 17:08:45.238497 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.238339    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ffd9bcfc9-zz9cf"
Jul 15 17:08:45.238497 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.238331    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ffd9bcfc9-zz9cf" event={"ID":"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c","Type":"ContainerDied","Data":"7bc9b4f48a8cd127ac80b346e0785d7a35ff6db039b7e29d846ea3c7b91de8e7"}
Jul 15 17:08:45.238497 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.238459    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ffd9bcfc9-zz9cf" event={"ID":"62d4bd49-f0c6-4bd5-bebf-1e68b96d780c","Type":"ContainerDied","Data":"0888714ea9280e9abcb999cdd33dd3a6829b8c41e524abce0a758af7712cbd12"}
Jul 15 17:08:45.238497 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.238480    2575 scope.go:117] "RemoveContainer" containerID="7bc9b4f48a8cd127ac80b346e0785d7a35ff6db039b7e29d846ea3c7b91de8e7"
Jul 15 17:08:45.247029 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.247014    2575 scope.go:117] "RemoveContainer" containerID="7bc9b4f48a8cd127ac80b346e0785d7a35ff6db039b7e29d846ea3c7b91de8e7"
Jul 15 17:08:45.247252 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:08:45.247235    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bc9b4f48a8cd127ac80b346e0785d7a35ff6db039b7e29d846ea3c7b91de8e7\": container with ID starting with 7bc9b4f48a8cd127ac80b346e0785d7a35ff6db039b7e29d846ea3c7b91de8e7 not found: ID does not exist" containerID="7bc9b4f48a8cd127ac80b346e0785d7a35ff6db039b7e29d846ea3c7b91de8e7"
Jul 15 17:08:45.247315 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.247260    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bc9b4f48a8cd127ac80b346e0785d7a35ff6db039b7e29d846ea3c7b91de8e7"} err="failed to get container status \"7bc9b4f48a8cd127ac80b346e0785d7a35ff6db039b7e29d846ea3c7b91de8e7\": rpc error: code = NotFound desc = could not find container \"7bc9b4f48a8cd127ac80b346e0785d7a35ff6db039b7e29d846ea3c7b91de8e7\": container with ID starting with 7bc9b4f48a8cd127ac80b346e0785d7a35ff6db039b7e29d846ea3c7b91de8e7 not found: ID does not exist"
Jul 15 17:08:45.259024 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.258997    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7ffd9bcfc9-zz9cf"]
Jul 15 17:08:45.272685 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:45.272661    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7ffd9bcfc9-zz9cf"]
Jul 15 17:08:46.525972 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:46.525941    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62d4bd49-f0c6-4bd5-bebf-1e68b96d780c" path="/var/lib/kubelet/pods/62d4bd49-f0c6-4bd5-bebf-1e68b96d780c/volumes"
Jul 15 17:08:50.213038 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:50.213008    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9p25"
Jul 15 17:08:58.343498 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.343464    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"]
Jul 15 17:08:58.343950 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.343809    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d2b50e4-b33f-49e1-9129-9d7d0158dc7f" containerName="manager"
Jul 15 17:08:58.343950 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.343821    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d2b50e4-b33f-49e1-9129-9d7d0158dc7f" containerName="manager"
Jul 15 17:08:58.343950 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.343849    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62d4bd49-f0c6-4bd5-bebf-1e68b96d780c" containerName="console"
Jul 15 17:08:58.343950 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.343855    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="62d4bd49-f0c6-4bd5-bebf-1e68b96d780c" containerName="console"
Jul 15 17:08:58.343950 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.343902    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="62d4bd49-f0c6-4bd5-bebf-1e68b96d780c" containerName="console"
Jul 15 17:08:58.343950 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.343915    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d2b50e4-b33f-49e1-9129-9d7d0158dc7f" containerName="manager"
Jul 15 17:08:58.346175 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.346157    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.348773 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.348752    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-bj5g7\""
Jul 15 17:08:58.360762 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.360740    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"]
Jul 15 17:08:58.528225 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.528190    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01190a7c-1f78-4e65-8582-73df15d76a5e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.528225 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.528228    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01190a7c-1f78-4e65-8582-73df15d76a5e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.528474 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.528260    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01190a7c-1f78-4e65-8582-73df15d76a5e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.528474 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.528279    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01190a7c-1f78-4e65-8582-73df15d76a5e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.528474 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.528332    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk5v5\" (UniqueName: \"kubernetes.io/projected/01190a7c-1f78-4e65-8582-73df15d76a5e-kube-api-access-tk5v5\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.528474 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.528376    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01190a7c-1f78-4e65-8582-73df15d76a5e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.528474 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.528407    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01190a7c-1f78-4e65-8582-73df15d76a5e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.528474 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.528429    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01190a7c-1f78-4e65-8582-73df15d76a5e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.528474 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.528461    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01190a7c-1f78-4e65-8582-73df15d76a5e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.629459 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.629418    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01190a7c-1f78-4e65-8582-73df15d76a5e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.629678 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.629467    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01190a7c-1f78-4e65-8582-73df15d76a5e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.629678 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.629512    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01190a7c-1f78-4e65-8582-73df15d76a5e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.629678 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.629534    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01190a7c-1f78-4e65-8582-73df15d76a5e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.629876 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.629678    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk5v5\" (UniqueName: \"kubernetes.io/projected/01190a7c-1f78-4e65-8582-73df15d76a5e-kube-api-access-tk5v5\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.629876 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.629731    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01190a7c-1f78-4e65-8582-73df15d76a5e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.629876 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.629765    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01190a7c-1f78-4e65-8582-73df15d76a5e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.629876 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.629793    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01190a7c-1f78-4e65-8582-73df15d76a5e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.629876 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.629825    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01190a7c-1f78-4e65-8582-73df15d76a5e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.630138 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.629964    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01190a7c-1f78-4e65-8582-73df15d76a5e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.630138 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.629994    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01190a7c-1f78-4e65-8582-73df15d76a5e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.630241 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.630153    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01190a7c-1f78-4e65-8582-73df15d76a5e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.630241 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.630221    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01190a7c-1f78-4e65-8582-73df15d76a5e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.630681 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.630653    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01190a7c-1f78-4e65-8582-73df15d76a5e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.631899 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.631883    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01190a7c-1f78-4e65-8582-73df15d76a5e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.632120 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.632102    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01190a7c-1f78-4e65-8582-73df15d76a5e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.639222 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.639200    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01190a7c-1f78-4e65-8582-73df15d76a5e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.639369 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.639352    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk5v5\" (UniqueName: \"kubernetes.io/projected/01190a7c-1f78-4e65-8582-73df15d76a5e-kube-api-access-tk5v5\") pod \"maas-default-gateway-openshift-default-56654dcf7-nhsnr\" (UID: \"01190a7c-1f78-4e65-8582-73df15d76a5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.656359 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.656330    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:58.812928 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.812904    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"]
Jul 15 17:08:58.814834 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:08:58.814806    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01190a7c_1f78_4e65_8582_73df15d76a5e.slice/crio-47cb6c47061837f20cdfdabbc42373967f28c784dabdb0d2e3d6fb6569b85c99 WatchSource:0}: Error finding container 47cb6c47061837f20cdfdabbc42373967f28c784dabdb0d2e3d6fb6569b85c99: Status 404 returned error can't find the container with id 47cb6c47061837f20cdfdabbc42373967f28c784dabdb0d2e3d6fb6569b85c99
Jul 15 17:08:58.816815 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.816777    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:08:58.816914 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.816851    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:08:58.816914 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:58.816895    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:08:59.303284 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:59.303248    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr" event={"ID":"01190a7c-1f78-4e65-8582-73df15d76a5e","Type":"ContainerStarted","Data":"80027fc86c07e0ac37dd131a234dfc192fea664d5a2c5c00762354995cb4d794"}
Jul 15 17:08:59.303462 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:59.303307    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr" event={"ID":"01190a7c-1f78-4e65-8582-73df15d76a5e","Type":"ContainerStarted","Data":"47cb6c47061837f20cdfdabbc42373967f28c784dabdb0d2e3d6fb6569b85c99"}
Jul 15 17:08:59.326010 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:59.325950    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr" podStartSLOduration=1.325930917 podStartE2EDuration="1.325930917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:08:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:08:59.324874182 +0000 UTC m=+755.342287291" watchObservedRunningTime="2026-07-15 17:08:59.325930917 +0000 UTC m=+755.343344014"
Jul 15 17:08:59.656525 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:59.656497    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:08:59.661117 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:08:59.661097    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:09:00.307610 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:00.307574    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:09:00.308362 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:00.308342    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:09:00.328490 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:00.328463    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hvmxq"]
Jul 15 17:09:00.330882 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:00.330865    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hvmxq"
Jul 15 17:09:00.335053 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:00.335032    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 15 17:09:00.347304 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:00.347282    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hvmxq"]
Jul 15 17:09:00.411057 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:00.411018    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hvmxq"]
Jul 15 17:09:00.446982 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:00.446953    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0c81b5fd-df9d-4d17-a91d-fd1ec889a788-config-file\") pod \"limitador-limitador-64c94fd64f-hvmxq\" (UID: \"0c81b5fd-df9d-4d17-a91d-fd1ec889a788\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hvmxq"
Jul 15 17:09:00.447228 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:00.447196    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5vgb\" (UniqueName: \"kubernetes.io/projected/0c81b5fd-df9d-4d17-a91d-fd1ec889a788-kube-api-access-z5vgb\") pod \"limitador-limitador-64c94fd64f-hvmxq\" (UID: \"0c81b5fd-df9d-4d17-a91d-fd1ec889a788\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hvmxq"
Jul 15 17:09:00.548474 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:00.548440    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0c81b5fd-df9d-4d17-a91d-fd1ec889a788-config-file\") pod \"limitador-limitador-64c94fd64f-hvmxq\" (UID: \"0c81b5fd-df9d-4d17-a91d-fd1ec889a788\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hvmxq"
Jul 15 17:09:00.548659 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:00.548599    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5vgb\" (UniqueName: \"kubernetes.io/projected/0c81b5fd-df9d-4d17-a91d-fd1ec889a788-kube-api-access-z5vgb\") pod \"limitador-limitador-64c94fd64f-hvmxq\" (UID: \"0c81b5fd-df9d-4d17-a91d-fd1ec889a788\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hvmxq"
Jul 15 17:09:00.549140 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:00.549118    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0c81b5fd-df9d-4d17-a91d-fd1ec889a788-config-file\") pod \"limitador-limitador-64c94fd64f-hvmxq\" (UID: \"0c81b5fd-df9d-4d17-a91d-fd1ec889a788\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hvmxq"
Jul 15 17:09:00.565001 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:00.564935    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5vgb\" (UniqueName: \"kubernetes.io/projected/0c81b5fd-df9d-4d17-a91d-fd1ec889a788-kube-api-access-z5vgb\") pod \"limitador-limitador-64c94fd64f-hvmxq\" (UID: \"0c81b5fd-df9d-4d17-a91d-fd1ec889a788\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hvmxq"
Jul 15 17:09:00.640497 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:00.640468    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hvmxq"
Jul 15 17:09:00.770780 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:00.770753    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hvmxq"]
Jul 15 17:09:00.773789 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:09:00.773755    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c81b5fd_df9d_4d17_a91d_fd1ec889a788.slice/crio-16223c8f5e582ade709d1045f7a28ffb762160c4c0f3c6f61e119ffcf42c820b WatchSource:0}: Error finding container 16223c8f5e582ade709d1045f7a28ffb762160c4c0f3c6f61e119ffcf42c820b: Status 404 returned error can't find the container with id 16223c8f5e582ade709d1045f7a28ffb762160c4c0f3c6f61e119ffcf42c820b
Jul 15 17:09:01.159316 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:01.159279    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-mbqg5"]
Jul 15 17:09:01.163594 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:01.163544    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mbqg5"
Jul 15 17:09:01.167070 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:01.167051    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jhq54\""
Jul 15 17:09:01.170761 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:01.170739    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mbqg5"]
Jul 15 17:09:01.254830 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:01.254802    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnwr9\" (UniqueName: \"kubernetes.io/projected/8b87c2c1-02f4-42bf-b951-5de3206b35e7-kube-api-access-pnwr9\") pod \"authorino-58cd955675-mbqg5\" (UID: \"8b87c2c1-02f4-42bf-b951-5de3206b35e7\") " pod="kuadrant-system/authorino-58cd955675-mbqg5"
Jul 15 17:09:01.312872 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:01.312830    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hvmxq" event={"ID":"0c81b5fd-df9d-4d17-a91d-fd1ec889a788","Type":"ContainerStarted","Data":"16223c8f5e582ade709d1045f7a28ffb762160c4c0f3c6f61e119ffcf42c820b"}
Jul 15 17:09:01.355247 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:01.355218    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnwr9\" (UniqueName: \"kubernetes.io/projected/8b87c2c1-02f4-42bf-b951-5de3206b35e7-kube-api-access-pnwr9\") pod \"authorino-58cd955675-mbqg5\" (UID: \"8b87c2c1-02f4-42bf-b951-5de3206b35e7\") " pod="kuadrant-system/authorino-58cd955675-mbqg5"
Jul 15 17:09:01.364285 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:01.364251    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnwr9\" (UniqueName: \"kubernetes.io/projected/8b87c2c1-02f4-42bf-b951-5de3206b35e7-kube-api-access-pnwr9\") pod \"authorino-58cd955675-mbqg5\" (UID: \"8b87c2c1-02f4-42bf-b951-5de3206b35e7\") " pod="kuadrant-system/authorino-58cd955675-mbqg5"
Jul 15 17:09:01.473913 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:01.473822    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mbqg5"
Jul 15 17:09:01.614227 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:01.614162    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mbqg5"]
Jul 15 17:09:01.617389 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:09:01.617358    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b87c2c1_02f4_42bf_b951_5de3206b35e7.slice/crio-7d8266af3ee541f4d951eb844a8a90797636a33be62180944ea2e33fbbe32fec WatchSource:0}: Error finding container 7d8266af3ee541f4d951eb844a8a90797636a33be62180944ea2e33fbbe32fec: Status 404 returned error can't find the container with id 7d8266af3ee541f4d951eb844a8a90797636a33be62180944ea2e33fbbe32fec
Jul 15 17:09:02.319077 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:02.319046    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mbqg5" event={"ID":"8b87c2c1-02f4-42bf-b951-5de3206b35e7","Type":"ContainerStarted","Data":"7d8266af3ee541f4d951eb844a8a90797636a33be62180944ea2e33fbbe32fec"}
Jul 15 17:09:04.331344 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:04.331311    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hvmxq" event={"ID":"0c81b5fd-df9d-4d17-a91d-fd1ec889a788","Type":"ContainerStarted","Data":"d3c5c1a704efa513d6f4c63fc759d13e3385f68896e590715cec04ee0634db78"}
Jul 15 17:09:04.331922 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:04.331439    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-hvmxq"
Jul 15 17:09:04.347574 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:04.347508    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-hvmxq" podStartSLOduration=1.469025768 podStartE2EDuration="4.347492097s" totalImagesPullingTime="2.878466329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:09:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:09:04.346573121 +0000 UTC m=+760.363986220" watchObservedRunningTime="2026-07-15 17:09:04.347492097 +0000 UTC m=+760.364905193"
Jul 15 17:09:05.140599 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:05.140551    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mbqg5"]
Jul 15 17:09:07.347643 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:07.347538    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mbqg5" event={"ID":"8b87c2c1-02f4-42bf-b951-5de3206b35e7","Type":"ContainerStarted","Data":"3be18b3ea81ca37d9239c53df1706d2db874582e3dfc4b40667cfbe159de3b62"}
Jul 15 17:09:07.347643 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:07.347548    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-mbqg5" podUID="8b87c2c1-02f4-42bf-b951-5de3206b35e7" containerName="authorino" containerID="cri-o://3be18b3ea81ca37d9239c53df1706d2db874582e3dfc4b40667cfbe159de3b62" gracePeriod=30
Jul 15 17:09:07.365962 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:07.365918    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-mbqg5" podStartSLOduration=1.111014328 podStartE2EDuration="6.365901429s" totalImagesPullingTime="5.254887101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:09:07.364966814 +0000 UTC m=+763.382379911" watchObservedRunningTime="2026-07-15 17:09:07.365901429 +0000 UTC m=+763.383314525"
Jul 15 17:09:07.595502 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:07.595479    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mbqg5"
Jul 15 17:09:07.712523 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:07.712499    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnwr9\" (UniqueName: \"kubernetes.io/projected/8b87c2c1-02f4-42bf-b951-5de3206b35e7-kube-api-access-pnwr9\") pod \"8b87c2c1-02f4-42bf-b951-5de3206b35e7\" (UID: \"8b87c2c1-02f4-42bf-b951-5de3206b35e7\") "
Jul 15 17:09:07.714464 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:07.714442    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b87c2c1-02f4-42bf-b951-5de3206b35e7-kube-api-access-pnwr9" (OuterVolumeSpecName: "kube-api-access-pnwr9") pod "8b87c2c1-02f4-42bf-b951-5de3206b35e7" (UID: "8b87c2c1-02f4-42bf-b951-5de3206b35e7"). InnerVolumeSpecName "kube-api-access-pnwr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:09:07.814052 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:07.814028    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnwr9\" (UniqueName: \"kubernetes.io/projected/8b87c2c1-02f4-42bf-b951-5de3206b35e7-kube-api-access-pnwr9\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:09:08.353579 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:08.353526    2575 generic.go:356] "Generic (PLEG): container finished" podID="8b87c2c1-02f4-42bf-b951-5de3206b35e7" containerID="3be18b3ea81ca37d9239c53df1706d2db874582e3dfc4b40667cfbe159de3b62" exitCode=0
Jul 15 17:09:08.354054 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:08.353585    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mbqg5"
Jul 15 17:09:08.354054 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:08.353596    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mbqg5" event={"ID":"8b87c2c1-02f4-42bf-b951-5de3206b35e7","Type":"ContainerDied","Data":"3be18b3ea81ca37d9239c53df1706d2db874582e3dfc4b40667cfbe159de3b62"}
Jul 15 17:09:08.354054 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:08.353640    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mbqg5" event={"ID":"8b87c2c1-02f4-42bf-b951-5de3206b35e7","Type":"ContainerDied","Data":"7d8266af3ee541f4d951eb844a8a90797636a33be62180944ea2e33fbbe32fec"}
Jul 15 17:09:08.354054 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:08.353662    2575 scope.go:117] "RemoveContainer" containerID="3be18b3ea81ca37d9239c53df1706d2db874582e3dfc4b40667cfbe159de3b62"
Jul 15 17:09:08.362448 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:08.362427    2575 scope.go:117] "RemoveContainer" containerID="3be18b3ea81ca37d9239c53df1706d2db874582e3dfc4b40667cfbe159de3b62"
Jul 15 17:09:08.363119 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:09:08.363100    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3be18b3ea81ca37d9239c53df1706d2db874582e3dfc4b40667cfbe159de3b62\": container with ID starting with 3be18b3ea81ca37d9239c53df1706d2db874582e3dfc4b40667cfbe159de3b62 not found: ID does not exist" containerID="3be18b3ea81ca37d9239c53df1706d2db874582e3dfc4b40667cfbe159de3b62"
Jul 15 17:09:08.363208 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:08.363127    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3be18b3ea81ca37d9239c53df1706d2db874582e3dfc4b40667cfbe159de3b62"} err="failed to get container status \"3be18b3ea81ca37d9239c53df1706d2db874582e3dfc4b40667cfbe159de3b62\": rpc error: code = NotFound desc = could not find container \"3be18b3ea81ca37d9239c53df1706d2db874582e3dfc4b40667cfbe159de3b62\": container with ID starting with 3be18b3ea81ca37d9239c53df1706d2db874582e3dfc4b40667cfbe159de3b62 not found: ID does not exist"
Jul 15 17:09:08.375510 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:08.375487    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mbqg5"]
Jul 15 17:09:08.378952 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:08.378931    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-mbqg5"]
Jul 15 17:09:08.525550 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:08.525518    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b87c2c1-02f4-42bf-b951-5de3206b35e7" path="/var/lib/kubelet/pods/8b87c2c1-02f4-42bf-b951-5de3206b35e7/volumes"
Jul 15 17:09:15.337167 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:15.337132    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-hvmxq"
Jul 15 17:09:15.436111 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:15.436083    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hvmxq"]
Jul 15 17:09:15.436277 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:15.436242    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-hvmxq" podUID="0c81b5fd-df9d-4d17-a91d-fd1ec889a788" containerName="limitador" containerID="cri-o://d3c5c1a704efa513d6f4c63fc759d13e3385f68896e590715cec04ee0634db78" gracePeriod=30
Jul 15 17:09:16.000660 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:16.000606    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hvmxq"
Jul 15 17:09:16.067849 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:16.067823    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5vgb\" (UniqueName: \"kubernetes.io/projected/0c81b5fd-df9d-4d17-a91d-fd1ec889a788-kube-api-access-z5vgb\") pod \"0c81b5fd-df9d-4d17-a91d-fd1ec889a788\" (UID: \"0c81b5fd-df9d-4d17-a91d-fd1ec889a788\") "
Jul 15 17:09:16.067977 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:16.067869    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0c81b5fd-df9d-4d17-a91d-fd1ec889a788-config-file\") pod \"0c81b5fd-df9d-4d17-a91d-fd1ec889a788\" (UID: \"0c81b5fd-df9d-4d17-a91d-fd1ec889a788\") "
Jul 15 17:09:16.068238 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:16.068215    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c81b5fd-df9d-4d17-a91d-fd1ec889a788-config-file" (OuterVolumeSpecName: "config-file") pod "0c81b5fd-df9d-4d17-a91d-fd1ec889a788" (UID: "0c81b5fd-df9d-4d17-a91d-fd1ec889a788"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:09:16.069778 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:16.069753    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c81b5fd-df9d-4d17-a91d-fd1ec889a788-kube-api-access-z5vgb" (OuterVolumeSpecName: "kube-api-access-z5vgb") pod "0c81b5fd-df9d-4d17-a91d-fd1ec889a788" (UID: "0c81b5fd-df9d-4d17-a91d-fd1ec889a788"). InnerVolumeSpecName "kube-api-access-z5vgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:09:16.168962 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:16.168920    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5vgb\" (UniqueName: \"kubernetes.io/projected/0c81b5fd-df9d-4d17-a91d-fd1ec889a788-kube-api-access-z5vgb\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:09:16.168962 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:16.168959    2575 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0c81b5fd-df9d-4d17-a91d-fd1ec889a788-config-file\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:09:16.390009 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:16.389975    2575 generic.go:356] "Generic (PLEG): container finished" podID="0c81b5fd-df9d-4d17-a91d-fd1ec889a788" containerID="d3c5c1a704efa513d6f4c63fc759d13e3385f68896e590715cec04ee0634db78" exitCode=0
Jul 15 17:09:16.390436 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:16.390019    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hvmxq"
Jul 15 17:09:16.390436 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:16.390034    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hvmxq" event={"ID":"0c81b5fd-df9d-4d17-a91d-fd1ec889a788","Type":"ContainerDied","Data":"d3c5c1a704efa513d6f4c63fc759d13e3385f68896e590715cec04ee0634db78"}
Jul 15 17:09:16.390436 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:16.390075    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hvmxq" event={"ID":"0c81b5fd-df9d-4d17-a91d-fd1ec889a788","Type":"ContainerDied","Data":"16223c8f5e582ade709d1045f7a28ffb762160c4c0f3c6f61e119ffcf42c820b"}
Jul 15 17:09:16.390436 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:16.390097    2575 scope.go:117] "RemoveContainer" containerID="d3c5c1a704efa513d6f4c63fc759d13e3385f68896e590715cec04ee0634db78"
Jul 15 17:09:16.397985 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:16.397968    2575 scope.go:117] "RemoveContainer" containerID="d3c5c1a704efa513d6f4c63fc759d13e3385f68896e590715cec04ee0634db78"
Jul 15 17:09:16.398211 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:09:16.398193    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3c5c1a704efa513d6f4c63fc759d13e3385f68896e590715cec04ee0634db78\": container with ID starting with d3c5c1a704efa513d6f4c63fc759d13e3385f68896e590715cec04ee0634db78 not found: ID does not exist" containerID="d3c5c1a704efa513d6f4c63fc759d13e3385f68896e590715cec04ee0634db78"
Jul 15 17:09:16.398257 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:16.398220    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3c5c1a704efa513d6f4c63fc759d13e3385f68896e590715cec04ee0634db78"} err="failed to get container status \"d3c5c1a704efa513d6f4c63fc759d13e3385f68896e590715cec04ee0634db78\": rpc error: code = NotFound desc = could not find container \"d3c5c1a704efa513d6f4c63fc759d13e3385f68896e590715cec04ee0634db78\": container with ID starting with d3c5c1a704efa513d6f4c63fc759d13e3385f68896e590715cec04ee0634db78 not found: ID does not exist"
Jul 15 17:09:16.411050 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:16.411030    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hvmxq"]
Jul 15 17:09:16.414807 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:16.414786    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hvmxq"]
Jul 15 17:09:16.525477 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:16.525441    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c81b5fd-df9d-4d17-a91d-fd1ec889a788" path="/var/lib/kubelet/pods/0c81b5fd-df9d-4d17-a91d-fd1ec889a788/volumes"
Jul 15 17:09:22.016341 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:22.016306    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-2zrsn"]
Jul 15 17:09:22.016830 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:22.016786    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c81b5fd-df9d-4d17-a91d-fd1ec889a788" containerName="limitador"
Jul 15 17:09:22.016830 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:22.016804    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c81b5fd-df9d-4d17-a91d-fd1ec889a788" containerName="limitador"
Jul 15 17:09:22.016945 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:22.016835    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b87c2c1-02f4-42bf-b951-5de3206b35e7" containerName="authorino"
Jul 15 17:09:22.016945 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:22.016845    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b87c2c1-02f4-42bf-b951-5de3206b35e7" containerName="authorino"
Jul 15 17:09:22.016945 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:22.016930    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c81b5fd-df9d-4d17-a91d-fd1ec889a788" containerName="limitador"
Jul 15 17:09:22.016945 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:22.016945    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b87c2c1-02f4-42bf-b951-5de3206b35e7" containerName="authorino"
Jul 15 17:09:22.019321 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:22.019294    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2zrsn"
Jul 15 17:09:22.021942 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:22.021921    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 17:09:22.022056 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:22.021948    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 17:09:22.022950 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:22.022930    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-wbdqw\""
Jul 15 17:09:22.023048 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:22.022974    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 15 17:09:22.028035 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:22.028012    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-2zrsn"]
Jul 15 17:09:22.107312 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:22.107280    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e35db717-b98d-4336-b4db-e1a411902f2f-data\") pod \"postgres-69fb4594cf-2zrsn\" (UID: \"e35db717-b98d-4336-b4db-e1a411902f2f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2zrsn"
Jul 15 17:09:22.107476 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:22.107356    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdkl2\" (UniqueName: \"kubernetes.io/projected/e35db717-b98d-4336-b4db-e1a411902f2f-kube-api-access-jdkl2\") pod \"postgres-69fb4594cf-2zrsn\" (UID: \"e35db717-b98d-4336-b4db-e1a411902f2f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2zrsn"
Jul 15 17:09:22.208658 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:22.208622    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdkl2\" (UniqueName: \"kubernetes.io/projected/e35db717-b98d-4336-b4db-e1a411902f2f-kube-api-access-jdkl2\") pod \"postgres-69fb4594cf-2zrsn\" (UID: \"e35db717-b98d-4336-b4db-e1a411902f2f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2zrsn"
Jul 15 17:09:22.208818 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:22.208664    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e35db717-b98d-4336-b4db-e1a411902f2f-data\") pod \"postgres-69fb4594cf-2zrsn\" (UID: \"e35db717-b98d-4336-b4db-e1a411902f2f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2zrsn"
Jul 15 17:09:22.208998 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:22.208982    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e35db717-b98d-4336-b4db-e1a411902f2f-data\") pod \"postgres-69fb4594cf-2zrsn\" (UID: \"e35db717-b98d-4336-b4db-e1a411902f2f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2zrsn"
Jul 15 17:09:22.216991 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:22.216970    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdkl2\" (UniqueName: \"kubernetes.io/projected/e35db717-b98d-4336-b4db-e1a411902f2f-kube-api-access-jdkl2\") pod \"postgres-69fb4594cf-2zrsn\" (UID: \"e35db717-b98d-4336-b4db-e1a411902f2f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2zrsn"
Jul 15 17:09:22.330314 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:22.330228    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2zrsn"
Jul 15 17:09:22.457957 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:22.457924    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-2zrsn"]
Jul 15 17:09:22.461490 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:09:22.461465    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode35db717_b98d_4336_b4db_e1a411902f2f.slice/crio-5194b3b515980281dbc2fe81bdda5573a8cd058c75c43b3fe1e0f2a55a41ef69 WatchSource:0}: Error finding container 5194b3b515980281dbc2fe81bdda5573a8cd058c75c43b3fe1e0f2a55a41ef69: Status 404 returned error can't find the container with id 5194b3b515980281dbc2fe81bdda5573a8cd058c75c43b3fe1e0f2a55a41ef69
Jul 15 17:09:23.427649 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:23.427601    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2zrsn" event={"ID":"e35db717-b98d-4336-b4db-e1a411902f2f","Type":"ContainerStarted","Data":"5194b3b515980281dbc2fe81bdda5573a8cd058c75c43b3fe1e0f2a55a41ef69"}
Jul 15 17:09:27.223881 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:27.223861    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 15 17:09:27.449665 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:27.449633    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2zrsn" event={"ID":"e35db717-b98d-4336-b4db-e1a411902f2f","Type":"ContainerStarted","Data":"1407a2a317e6c8ae21ff891b829998c43bf59a9d4afbbba71d7599dc260fde7a"}
Jul 15 17:09:27.449811 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:27.449763    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2zrsn"
Jul 15 17:09:27.467975 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:27.467929    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2zrsn" podStartSLOduration=1.709521971 podStartE2EDuration="6.467914043s" totalImagesPullingTime="4.758392072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:09:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:09:27.465791687 +0000 UTC m=+783.483204807" watchObservedRunningTime="2026-07-15 17:09:27.467914043 +0000 UTC m=+783.485327140"
Jul 15 17:09:33.483206 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:33.483175    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2zrsn"
Jul 15 17:09:34.042642 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.042609    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-z5cmf"]
Jul 15 17:09:34.048616 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.048593    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-z5cmf"
Jul 15 17:09:34.050846 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.050825    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jhq54\""
Jul 15 17:09:34.053658 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.053637    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-z5cmf"]
Jul 15 17:09:34.114744 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.114713    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dkbl\" (UniqueName: \"kubernetes.io/projected/ad37fd62-9509-4d27-b17d-adfc5376da6e-kube-api-access-8dkbl\") pod \"authorino-5d4b67c446-z5cmf\" (UID: \"ad37fd62-9509-4d27-b17d-adfc5376da6e\") " pod="kuadrant-system/authorino-5d4b67c446-z5cmf"
Jul 15 17:09:34.215979 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.215944    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dkbl\" (UniqueName: \"kubernetes.io/projected/ad37fd62-9509-4d27-b17d-adfc5376da6e-kube-api-access-8dkbl\") pod \"authorino-5d4b67c446-z5cmf\" (UID: \"ad37fd62-9509-4d27-b17d-adfc5376da6e\") " pod="kuadrant-system/authorino-5d4b67c446-z5cmf"
Jul 15 17:09:34.224702 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.224678    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dkbl\" (UniqueName: \"kubernetes.io/projected/ad37fd62-9509-4d27-b17d-adfc5376da6e-kube-api-access-8dkbl\") pod \"authorino-5d4b67c446-z5cmf\" (UID: \"ad37fd62-9509-4d27-b17d-adfc5376da6e\") " pod="kuadrant-system/authorino-5d4b67c446-z5cmf"
Jul 15 17:09:34.291316 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.291287    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-z5cmf"]
Jul 15 17:09:34.291533 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.291522    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-z5cmf"
Jul 15 17:09:34.315627 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.315597    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-cd479df96-q2p8z"]
Jul 15 17:09:34.320469 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.320446    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cd479df96-q2p8z"
Jul 15 17:09:34.325487 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.325460    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cd479df96-q2p8z"]
Jul 15 17:09:34.409818 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.409788    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cd479df96-q2p8z"]
Jul 15 17:09:34.410072 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:09:34.410054    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-gkbv4], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-cd479df96-q2p8z" podUID="8a6a5c99-f2fb-4f50-a302-09f05a8e52e0"
Jul 15 17:09:34.417933 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.417901    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkbv4\" (UniqueName: \"kubernetes.io/projected/8a6a5c99-f2fb-4f50-a302-09f05a8e52e0-kube-api-access-gkbv4\") pod \"authorino-cd479df96-q2p8z\" (UID: \"8a6a5c99-f2fb-4f50-a302-09f05a8e52e0\") " pod="kuadrant-system/authorino-cd479df96-q2p8z"
Jul 15 17:09:34.431291 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.431265    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-z5cmf"]
Jul 15 17:09:34.434744 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:09:34.434720    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad37fd62_9509_4d27_b17d_adfc5376da6e.slice/crio-f0cbb0abe0f82021bf3ba88079e8fe8b57c0a89e8101e5f01a8c33a429c5f4ce WatchSource:0}: Error finding container f0cbb0abe0f82021bf3ba88079e8fe8b57c0a89e8101e5f01a8c33a429c5f4ce: Status 404 returned error can't find the container with id f0cbb0abe0f82021bf3ba88079e8fe8b57c0a89e8101e5f01a8c33a429c5f4ce
Jul 15 17:09:34.482874 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.482849    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cd479df96-q2p8z"
Jul 15 17:09:34.482874 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.482850    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-z5cmf" event={"ID":"ad37fd62-9509-4d27-b17d-adfc5376da6e","Type":"ContainerStarted","Data":"f0cbb0abe0f82021bf3ba88079e8fe8b57c0a89e8101e5f01a8c33a429c5f4ce"}
Jul 15 17:09:34.490572 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.490536    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cd479df96-q2p8z"
Jul 15 17:09:34.518608 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.518578    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkbv4\" (UniqueName: \"kubernetes.io/projected/8a6a5c99-f2fb-4f50-a302-09f05a8e52e0-kube-api-access-gkbv4\") pod \"authorino-cd479df96-q2p8z\" (UID: \"8a6a5c99-f2fb-4f50-a302-09f05a8e52e0\") " pod="kuadrant-system/authorino-cd479df96-q2p8z"
Jul 15 17:09:34.526466 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.526443    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkbv4\" (UniqueName: \"kubernetes.io/projected/8a6a5c99-f2fb-4f50-a302-09f05a8e52e0-kube-api-access-gkbv4\") pod \"authorino-cd479df96-q2p8z\" (UID: \"8a6a5c99-f2fb-4f50-a302-09f05a8e52e0\") " pod="kuadrant-system/authorino-cd479df96-q2p8z"
Jul 15 17:09:34.619282 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.619259    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkbv4\" (UniqueName: \"kubernetes.io/projected/8a6a5c99-f2fb-4f50-a302-09f05a8e52e0-kube-api-access-gkbv4\") pod \"8a6a5c99-f2fb-4f50-a302-09f05a8e52e0\" (UID: \"8a6a5c99-f2fb-4f50-a302-09f05a8e52e0\") "
Jul 15 17:09:34.621327 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.621295    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a6a5c99-f2fb-4f50-a302-09f05a8e52e0-kube-api-access-gkbv4" (OuterVolumeSpecName: "kube-api-access-gkbv4") pod "8a6a5c99-f2fb-4f50-a302-09f05a8e52e0" (UID: "8a6a5c99-f2fb-4f50-a302-09f05a8e52e0"). InnerVolumeSpecName "kube-api-access-gkbv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:09:34.720310 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:34.720282    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkbv4\" (UniqueName: \"kubernetes.io/projected/8a6a5c99-f2fb-4f50-a302-09f05a8e52e0-kube-api-access-gkbv4\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:09:35.488916 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:35.488837    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cd479df96-q2p8z"
Jul 15 17:09:35.488916 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:35.488845    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-z5cmf" podUID="ad37fd62-9509-4d27-b17d-adfc5376da6e" containerName="authorino" containerID="cri-o://51c8fd0784b157155191240dee4325af1a82310faa203e4a329dbb0fdec1e8c9" gracePeriod=30
Jul 15 17:09:35.488916 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:35.488837    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-z5cmf" event={"ID":"ad37fd62-9509-4d27-b17d-adfc5376da6e","Type":"ContainerStarted","Data":"51c8fd0784b157155191240dee4325af1a82310faa203e4a329dbb0fdec1e8c9"}
Jul 15 17:09:35.505785 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:35.505743    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-z5cmf" podStartSLOduration=1.204498412 podStartE2EDuration="1.505730954s" totalImagesPullingTime="301.232542ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:09:35.504318377 +0000 UTC m=+791.521731492" watchObservedRunningTime="2026-07-15 17:09:35.505730954 +0000 UTC m=+791.523144050"
Jul 15 17:09:35.527744 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:35.527721    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cd479df96-q2p8z"]
Jul 15 17:09:35.534550 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:35.532102    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-cd479df96-q2p8z"]
Jul 15 17:09:35.742871 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:35.742816    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-z5cmf"
Jul 15 17:09:35.830596 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:35.830546    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dkbl\" (UniqueName: \"kubernetes.io/projected/ad37fd62-9509-4d27-b17d-adfc5376da6e-kube-api-access-8dkbl\") pod \"ad37fd62-9509-4d27-b17d-adfc5376da6e\" (UID: \"ad37fd62-9509-4d27-b17d-adfc5376da6e\") "
Jul 15 17:09:35.832514 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:35.832482    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad37fd62-9509-4d27-b17d-adfc5376da6e-kube-api-access-8dkbl" (OuterVolumeSpecName: "kube-api-access-8dkbl") pod "ad37fd62-9509-4d27-b17d-adfc5376da6e" (UID: "ad37fd62-9509-4d27-b17d-adfc5376da6e"). InnerVolumeSpecName "kube-api-access-8dkbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:09:35.931959 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:35.931927    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dkbl\" (UniqueName: \"kubernetes.io/projected/ad37fd62-9509-4d27-b17d-adfc5376da6e-kube-api-access-8dkbl\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:09:36.495230 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:36.495182    2575 generic.go:356] "Generic (PLEG): container finished" podID="ad37fd62-9509-4d27-b17d-adfc5376da6e" containerID="51c8fd0784b157155191240dee4325af1a82310faa203e4a329dbb0fdec1e8c9" exitCode=0
Jul 15 17:09:36.495230 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:36.495228    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-z5cmf"
Jul 15 17:09:36.495467 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:36.495249    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-z5cmf" event={"ID":"ad37fd62-9509-4d27-b17d-adfc5376da6e","Type":"ContainerDied","Data":"51c8fd0784b157155191240dee4325af1a82310faa203e4a329dbb0fdec1e8c9"}
Jul 15 17:09:36.495467 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:36.495305    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-z5cmf" event={"ID":"ad37fd62-9509-4d27-b17d-adfc5376da6e","Type":"ContainerDied","Data":"f0cbb0abe0f82021bf3ba88079e8fe8b57c0a89e8101e5f01a8c33a429c5f4ce"}
Jul 15 17:09:36.495467 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:36.495328    2575 scope.go:117] "RemoveContainer" containerID="51c8fd0784b157155191240dee4325af1a82310faa203e4a329dbb0fdec1e8c9"
Jul 15 17:09:36.503774 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:36.503756    2575 scope.go:117] "RemoveContainer" containerID="51c8fd0784b157155191240dee4325af1a82310faa203e4a329dbb0fdec1e8c9"
Jul 15 17:09:36.503998 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:09:36.503979    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51c8fd0784b157155191240dee4325af1a82310faa203e4a329dbb0fdec1e8c9\": container with ID starting with 51c8fd0784b157155191240dee4325af1a82310faa203e4a329dbb0fdec1e8c9 not found: ID does not exist" containerID="51c8fd0784b157155191240dee4325af1a82310faa203e4a329dbb0fdec1e8c9"
Jul 15 17:09:36.504050 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:36.504004    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51c8fd0784b157155191240dee4325af1a82310faa203e4a329dbb0fdec1e8c9"} err="failed to get container status \"51c8fd0784b157155191240dee4325af1a82310faa203e4a329dbb0fdec1e8c9\": rpc error: code = NotFound desc = could not find container \"51c8fd0784b157155191240dee4325af1a82310faa203e4a329dbb0fdec1e8c9\": container with ID starting with 51c8fd0784b157155191240dee4325af1a82310faa203e4a329dbb0fdec1e8c9 not found: ID does not exist"
Jul 15 17:09:36.515586 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:36.515544    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-z5cmf"]
Jul 15 17:09:36.519276 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:36.519258    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-z5cmf"]
Jul 15 17:09:36.525738 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:36.525717    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a6a5c99-f2fb-4f50-a302-09f05a8e52e0" path="/var/lib/kubelet/pods/8a6a5c99-f2fb-4f50-a302-09f05a8e52e0/volumes"
Jul 15 17:09:36.525942 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:36.525930    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad37fd62-9509-4d27-b17d-adfc5376da6e" path="/var/lib/kubelet/pods/ad37fd62-9509-4d27-b17d-adfc5376da6e/volumes"
Jul 15 17:09:42.934262 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:42.934228    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7447546df6-tvfcd"]
Jul 15 17:09:42.934712 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:42.934608    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad37fd62-9509-4d27-b17d-adfc5376da6e" containerName="authorino"
Jul 15 17:09:42.934712 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:42.934620    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad37fd62-9509-4d27-b17d-adfc5376da6e" containerName="authorino"
Jul 15 17:09:42.934712 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:42.934677    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad37fd62-9509-4d27-b17d-adfc5376da6e" containerName="authorino"
Jul 15 17:09:42.956918 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:42.956882    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7447546df6-tvfcd"]
Jul 15 17:09:42.957066 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:42.957009    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7447546df6-tvfcd"
Jul 15 17:09:42.959567 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:42.959532    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nr2cc\""
Jul 15 17:09:42.959693 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:42.959588    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 15 17:09:42.959693 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:42.959598    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 15 17:09:43.093117 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:43.093079    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eba5b26d-074b-446e-ac4b-f832d5a50afc-webhook-cert\") pod \"maas-controller-7447546df6-tvfcd\" (UID: \"eba5b26d-074b-446e-ac4b-f832d5a50afc\") " pod="opendatahub/maas-controller-7447546df6-tvfcd"
Jul 15 17:09:43.093309 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:43.093193    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcq8p\" (UniqueName: \"kubernetes.io/projected/eba5b26d-074b-446e-ac4b-f832d5a50afc-kube-api-access-fcq8p\") pod \"maas-controller-7447546df6-tvfcd\" (UID: \"eba5b26d-074b-446e-ac4b-f832d5a50afc\") " pod="opendatahub/maas-controller-7447546df6-tvfcd"
Jul 15 17:09:43.121086 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:43.121055    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b8d7f4fcb-pb8rg"]
Jul 15 17:09:43.123483 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:43.123467    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b8d7f4fcb-pb8rg"
Jul 15 17:09:43.134059 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:43.134037    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b8d7f4fcb-pb8rg"]
Jul 15 17:09:43.194738 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:43.194656    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eba5b26d-074b-446e-ac4b-f832d5a50afc-webhook-cert\") pod \"maas-controller-7447546df6-tvfcd\" (UID: \"eba5b26d-074b-446e-ac4b-f832d5a50afc\") " pod="opendatahub/maas-controller-7447546df6-tvfcd"
Jul 15 17:09:43.194738 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:43.194707    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d6d032c-dc68-4402-9151-d6a18d905d02-webhook-cert\") pod \"maas-controller-6b8d7f4fcb-pb8rg\" (UID: \"9d6d032c-dc68-4402-9151-d6a18d905d02\") " pod="opendatahub/maas-controller-6b8d7f4fcb-pb8rg"
Jul 15 17:09:43.194738 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:43.194730    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl7m4\" (UniqueName: \"kubernetes.io/projected/9d6d032c-dc68-4402-9151-d6a18d905d02-kube-api-access-dl7m4\") pod \"maas-controller-6b8d7f4fcb-pb8rg\" (UID: \"9d6d032c-dc68-4402-9151-d6a18d905d02\") " pod="opendatahub/maas-controller-6b8d7f4fcb-pb8rg"
Jul 15 17:09:43.194946 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:43.194756    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcq8p\" (UniqueName: \"kubernetes.io/projected/eba5b26d-074b-446e-ac4b-f832d5a50afc-kube-api-access-fcq8p\") pod \"maas-controller-7447546df6-tvfcd\" (UID: \"eba5b26d-074b-446e-ac4b-f832d5a50afc\") " pod="opendatahub/maas-controller-7447546df6-tvfcd"
Jul 15 17:09:43.197095 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:43.197074    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eba5b26d-074b-446e-ac4b-f832d5a50afc-webhook-cert\") pod \"maas-controller-7447546df6-tvfcd\" (UID: \"eba5b26d-074b-446e-ac4b-f832d5a50afc\") " pod="opendatahub/maas-controller-7447546df6-tvfcd"
Jul 15 17:09:43.202739 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:43.202718    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcq8p\" (UniqueName: \"kubernetes.io/projected/eba5b26d-074b-446e-ac4b-f832d5a50afc-kube-api-access-fcq8p\") pod \"maas-controller-7447546df6-tvfcd\" (UID: \"eba5b26d-074b-446e-ac4b-f832d5a50afc\") " pod="opendatahub/maas-controller-7447546df6-tvfcd"
Jul 15 17:09:43.267825 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:43.267800    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7447546df6-tvfcd"
Jul 15 17:09:43.295865 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:43.295836    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d6d032c-dc68-4402-9151-d6a18d905d02-webhook-cert\") pod \"maas-controller-6b8d7f4fcb-pb8rg\" (UID: \"9d6d032c-dc68-4402-9151-d6a18d905d02\") " pod="opendatahub/maas-controller-6b8d7f4fcb-pb8rg"
Jul 15 17:09:43.295994 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:43.295881    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl7m4\" (UniqueName: \"kubernetes.io/projected/9d6d032c-dc68-4402-9151-d6a18d905d02-kube-api-access-dl7m4\") pod \"maas-controller-6b8d7f4fcb-pb8rg\" (UID: \"9d6d032c-dc68-4402-9151-d6a18d905d02\") " pod="opendatahub/maas-controller-6b8d7f4fcb-pb8rg"
Jul 15 17:09:43.298250 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:43.298225    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d6d032c-dc68-4402-9151-d6a18d905d02-webhook-cert\") pod \"maas-controller-6b8d7f4fcb-pb8rg\" (UID: \"9d6d032c-dc68-4402-9151-d6a18d905d02\") " pod="opendatahub/maas-controller-6b8d7f4fcb-pb8rg"
Jul 15 17:09:43.303795 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:43.303769    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl7m4\" (UniqueName: \"kubernetes.io/projected/9d6d032c-dc68-4402-9151-d6a18d905d02-kube-api-access-dl7m4\") pod \"maas-controller-6b8d7f4fcb-pb8rg\" (UID: \"9d6d032c-dc68-4402-9151-d6a18d905d02\") " pod="opendatahub/maas-controller-6b8d7f4fcb-pb8rg"
Jul 15 17:09:43.394342 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:43.394285    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7447546df6-tvfcd"]
Jul 15 17:09:43.396513 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:09:43.396488    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeba5b26d_074b_446e_ac4b_f832d5a50afc.slice/crio-20e1329e4cf0494729e88be929399c7d2f9cdf5180a53759316b1159a58cb3c2 WatchSource:0}: Error finding container 20e1329e4cf0494729e88be929399c7d2f9cdf5180a53759316b1159a58cb3c2: Status 404 returned error can't find the container with id 20e1329e4cf0494729e88be929399c7d2f9cdf5180a53759316b1159a58cb3c2
Jul 15 17:09:43.433931 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:43.433904    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b8d7f4fcb-pb8rg"
Jul 15 17:09:43.533867 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:43.533835    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7447546df6-tvfcd" event={"ID":"eba5b26d-074b-446e-ac4b-f832d5a50afc","Type":"ContainerStarted","Data":"20e1329e4cf0494729e88be929399c7d2f9cdf5180a53759316b1159a58cb3c2"}
Jul 15 17:09:43.558282 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:43.558259    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b8d7f4fcb-pb8rg"]
Jul 15 17:09:43.560795 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:09:43.560770    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d6d032c_dc68_4402_9151_d6a18d905d02.slice/crio-01a95c8aad67d70d375ba711b6e6134f5f031761e56de0bbfc63b1ed2ba77136 WatchSource:0}: Error finding container 01a95c8aad67d70d375ba711b6e6134f5f031761e56de0bbfc63b1ed2ba77136: Status 404 returned error can't find the container with id 01a95c8aad67d70d375ba711b6e6134f5f031761e56de0bbfc63b1ed2ba77136
Jul 15 17:09:44.539323 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:44.539285    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b8d7f4fcb-pb8rg" event={"ID":"9d6d032c-dc68-4402-9151-d6a18d905d02","Type":"ContainerStarted","Data":"01a95c8aad67d70d375ba711b6e6134f5f031761e56de0bbfc63b1ed2ba77136"}
Jul 15 17:09:46.550697 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:46.550592    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b8d7f4fcb-pb8rg" event={"ID":"9d6d032c-dc68-4402-9151-d6a18d905d02","Type":"ContainerStarted","Data":"8e65f384a9e01dbd734884c75ca56e90c26c5ed18e8c1b4d3b07ba12a71d1d53"}
Jul 15 17:09:46.550697 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:46.550660    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b8d7f4fcb-pb8rg"
Jul 15 17:09:46.551875 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:46.551853    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7447546df6-tvfcd" event={"ID":"eba5b26d-074b-446e-ac4b-f832d5a50afc","Type":"ContainerStarted","Data":"f3ba0261db5d9daf030eacab6be3fe8246feb996f0103ab98f1aef6c8d165086"}
Jul 15 17:09:46.551993 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:46.551978    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7447546df6-tvfcd"
Jul 15 17:09:46.579688 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:46.579640    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6b8d7f4fcb-pb8rg" podStartSLOduration=0.981387362 podStartE2EDuration="3.579623963s" totalImagesPullingTime="2.598236601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:09:46.577796683 +0000 UTC m=+802.595209795" watchObservedRunningTime="2026-07-15 17:09:46.579623963 +0000 UTC m=+802.597037070"
Jul 15 17:09:46.619507 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:46.619463    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7447546df6-tvfcd" podStartSLOduration=1.8577284029999999 podStartE2EDuration="4.619447297s" totalImagesPullingTime="2.761718894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:09:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:09:46.613734003 +0000 UTC m=+802.631147102" watchObservedRunningTime="2026-07-15 17:09:46.619447297 +0000 UTC m=+802.636860392"
Jul 15 17:09:57.558629 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:57.558601    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7447546df6-tvfcd"
Jul 15 17:09:57.559004 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:57.558669    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b8d7f4fcb-pb8rg"
Jul 15 17:09:57.620382 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:57.620355    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7447546df6-tvfcd"]
Jul 15 17:09:57.620602 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:57.620575    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7447546df6-tvfcd" podUID="eba5b26d-074b-446e-ac4b-f832d5a50afc" containerName="manager" containerID="cri-o://f3ba0261db5d9daf030eacab6be3fe8246feb996f0103ab98f1aef6c8d165086" gracePeriod=30
Jul 15 17:09:57.865706 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:57.865683    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7447546df6-tvfcd"
Jul 15 17:09:58.023059 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:58.023029    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eba5b26d-074b-446e-ac4b-f832d5a50afc-webhook-cert\") pod \"eba5b26d-074b-446e-ac4b-f832d5a50afc\" (UID: \"eba5b26d-074b-446e-ac4b-f832d5a50afc\") "
Jul 15 17:09:58.023233 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:58.023071    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcq8p\" (UniqueName: \"kubernetes.io/projected/eba5b26d-074b-446e-ac4b-f832d5a50afc-kube-api-access-fcq8p\") pod \"eba5b26d-074b-446e-ac4b-f832d5a50afc\" (UID: \"eba5b26d-074b-446e-ac4b-f832d5a50afc\") "
Jul 15 17:09:58.025206 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:58.025181    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eba5b26d-074b-446e-ac4b-f832d5a50afc-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "eba5b26d-074b-446e-ac4b-f832d5a50afc" (UID: "eba5b26d-074b-446e-ac4b-f832d5a50afc"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:09:58.025331 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:58.025312    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eba5b26d-074b-446e-ac4b-f832d5a50afc-kube-api-access-fcq8p" (OuterVolumeSpecName: "kube-api-access-fcq8p") pod "eba5b26d-074b-446e-ac4b-f832d5a50afc" (UID: "eba5b26d-074b-446e-ac4b-f832d5a50afc"). InnerVolumeSpecName "kube-api-access-fcq8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:09:58.123574 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:58.123532    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcq8p\" (UniqueName: \"kubernetes.io/projected/eba5b26d-074b-446e-ac4b-f832d5a50afc-kube-api-access-fcq8p\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:09:58.123574 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:58.123573    2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eba5b26d-074b-446e-ac4b-f832d5a50afc-webhook-cert\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:09:58.604341 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:58.604309    2575 generic.go:356] "Generic (PLEG): container finished" podID="eba5b26d-074b-446e-ac4b-f832d5a50afc" containerID="f3ba0261db5d9daf030eacab6be3fe8246feb996f0103ab98f1aef6c8d165086" exitCode=0
Jul 15 17:09:58.604777 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:58.604358    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7447546df6-tvfcd"
Jul 15 17:09:58.604777 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:58.604387    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7447546df6-tvfcd" event={"ID":"eba5b26d-074b-446e-ac4b-f832d5a50afc","Type":"ContainerDied","Data":"f3ba0261db5d9daf030eacab6be3fe8246feb996f0103ab98f1aef6c8d165086"}
Jul 15 17:09:58.604777 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:58.604438    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7447546df6-tvfcd" event={"ID":"eba5b26d-074b-446e-ac4b-f832d5a50afc","Type":"ContainerDied","Data":"20e1329e4cf0494729e88be929399c7d2f9cdf5180a53759316b1159a58cb3c2"}
Jul 15 17:09:58.604777 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:58.604458    2575 scope.go:117] "RemoveContainer" containerID="f3ba0261db5d9daf030eacab6be3fe8246feb996f0103ab98f1aef6c8d165086"
Jul 15 17:09:58.613264 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:58.613249    2575 scope.go:117] "RemoveContainer" containerID="f3ba0261db5d9daf030eacab6be3fe8246feb996f0103ab98f1aef6c8d165086"
Jul 15 17:09:58.613495 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:09:58.613477    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3ba0261db5d9daf030eacab6be3fe8246feb996f0103ab98f1aef6c8d165086\": container with ID starting with f3ba0261db5d9daf030eacab6be3fe8246feb996f0103ab98f1aef6c8d165086 not found: ID does not exist" containerID="f3ba0261db5d9daf030eacab6be3fe8246feb996f0103ab98f1aef6c8d165086"
Jul 15 17:09:58.613569 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:58.613503    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3ba0261db5d9daf030eacab6be3fe8246feb996f0103ab98f1aef6c8d165086"} err="failed to get container status \"f3ba0261db5d9daf030eacab6be3fe8246feb996f0103ab98f1aef6c8d165086\": rpc error: code = NotFound desc = could not find container \"f3ba0261db5d9daf030eacab6be3fe8246feb996f0103ab98f1aef6c8d165086\": container with ID starting with f3ba0261db5d9daf030eacab6be3fe8246feb996f0103ab98f1aef6c8d165086 not found: ID does not exist"
Jul 15 17:09:58.621742 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:58.621719    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7447546df6-tvfcd"]
Jul 15 17:09:58.625531 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:09:58.625510    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7447546df6-tvfcd"]
Jul 15 17:10:00.527008 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:00.526969    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eba5b26d-074b-446e-ac4b-f832d5a50afc" path="/var/lib/kubelet/pods/eba5b26d-074b-446e-ac4b-f832d5a50afc/volumes"
Jul 15 17:10:20.092599 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.092552    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7f869c5dd8-t4f6b"]
Jul 15 17:10:20.093060 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.092883    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eba5b26d-074b-446e-ac4b-f832d5a50afc" containerName="manager"
Jul 15 17:10:20.093060 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.092895    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="eba5b26d-074b-446e-ac4b-f832d5a50afc" containerName="manager"
Jul 15 17:10:20.093060 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.092967    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="eba5b26d-074b-446e-ac4b-f832d5a50afc" containerName="manager"
Jul 15 17:10:20.095177 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.095157    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f869c5dd8-t4f6b"
Jul 15 17:10:20.097658 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.097638    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 15 17:10:20.097757 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.097639    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-77cxl\""
Jul 15 17:10:20.104303 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.104280    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f869c5dd8-t4f6b"]
Jul 15 17:10:20.206877 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.206846    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89b150b3-cdf8-4bc2-80db-03a6c7748aaa-openshift-service-ca\") pod \"maas-api-7f869c5dd8-t4f6b\" (UID: \"89b150b3-cdf8-4bc2-80db-03a6c7748aaa\") " pod="odh-ai-gateway-infra/maas-api-7f869c5dd8-t4f6b"
Jul 15 17:10:20.207015 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.206894    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsgxm\" (UniqueName: \"kubernetes.io/projected/89b150b3-cdf8-4bc2-80db-03a6c7748aaa-kube-api-access-jsgxm\") pod \"maas-api-7f869c5dd8-t4f6b\" (UID: \"89b150b3-cdf8-4bc2-80db-03a6c7748aaa\") " pod="odh-ai-gateway-infra/maas-api-7f869c5dd8-t4f6b"
Jul 15 17:10:20.207015 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.206964    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89b150b3-cdf8-4bc2-80db-03a6c7748aaa-maas-api-tls\") pod \"maas-api-7f869c5dd8-t4f6b\" (UID: \"89b150b3-cdf8-4bc2-80db-03a6c7748aaa\") " pod="odh-ai-gateway-infra/maas-api-7f869c5dd8-t4f6b"
Jul 15 17:10:20.307785 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.307754    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89b150b3-cdf8-4bc2-80db-03a6c7748aaa-openshift-service-ca\") pod \"maas-api-7f869c5dd8-t4f6b\" (UID: \"89b150b3-cdf8-4bc2-80db-03a6c7748aaa\") " pod="odh-ai-gateway-infra/maas-api-7f869c5dd8-t4f6b"
Jul 15 17:10:20.307953 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.307812    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsgxm\" (UniqueName: \"kubernetes.io/projected/89b150b3-cdf8-4bc2-80db-03a6c7748aaa-kube-api-access-jsgxm\") pod \"maas-api-7f869c5dd8-t4f6b\" (UID: \"89b150b3-cdf8-4bc2-80db-03a6c7748aaa\") " pod="odh-ai-gateway-infra/maas-api-7f869c5dd8-t4f6b"
Jul 15 17:10:20.307953 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.307878    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89b150b3-cdf8-4bc2-80db-03a6c7748aaa-maas-api-tls\") pod \"maas-api-7f869c5dd8-t4f6b\" (UID: \"89b150b3-cdf8-4bc2-80db-03a6c7748aaa\") " pod="odh-ai-gateway-infra/maas-api-7f869c5dd8-t4f6b"
Jul 15 17:10:20.308498 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.308476    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89b150b3-cdf8-4bc2-80db-03a6c7748aaa-openshift-service-ca\") pod \"maas-api-7f869c5dd8-t4f6b\" (UID: \"89b150b3-cdf8-4bc2-80db-03a6c7748aaa\") " pod="odh-ai-gateway-infra/maas-api-7f869c5dd8-t4f6b"
Jul 15 17:10:20.310288 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.310270    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89b150b3-cdf8-4bc2-80db-03a6c7748aaa-maas-api-tls\") pod \"maas-api-7f869c5dd8-t4f6b\" (UID: \"89b150b3-cdf8-4bc2-80db-03a6c7748aaa\") " pod="odh-ai-gateway-infra/maas-api-7f869c5dd8-t4f6b"
Jul 15 17:10:20.316191 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.316168    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsgxm\" (UniqueName: \"kubernetes.io/projected/89b150b3-cdf8-4bc2-80db-03a6c7748aaa-kube-api-access-jsgxm\") pod \"maas-api-7f869c5dd8-t4f6b\" (UID: \"89b150b3-cdf8-4bc2-80db-03a6c7748aaa\") " pod="odh-ai-gateway-infra/maas-api-7f869c5dd8-t4f6b"
Jul 15 17:10:20.406059 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.406029    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f869c5dd8-t4f6b"
Jul 15 17:10:20.546859 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.546833    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f869c5dd8-t4f6b"]
Jul 15 17:10:20.549288 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:10:20.549257    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89b150b3_cdf8_4bc2_80db_03a6c7748aaa.slice/crio-ba9e1d2017d964237943de8fad99745e76d728fcb91d16efed7aba007a10fc47 WatchSource:0}: Error finding container ba9e1d2017d964237943de8fad99745e76d728fcb91d16efed7aba007a10fc47: Status 404 returned error can't find the container with id ba9e1d2017d964237943de8fad99745e76d728fcb91d16efed7aba007a10fc47
Jul 15 17:10:20.648057 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.648024    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-ngw49"]
Jul 15 17:10:20.651003 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.650976    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-ngw49"
Jul 15 17:10:20.653540 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.653517    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-pkhjr\""
Jul 15 17:10:20.653661 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.653520    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 15 17:10:20.653919 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.653896    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-7f7qf"]
Jul 15 17:10:20.656472 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.656428    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-7f7qf"
Jul 15 17:10:20.661410 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.661392    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-ngw49"]
Jul 15 17:10:20.667059 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.667032    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-7f7qf"]
Jul 15 17:10:20.700603 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.700576    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f869c5dd8-t4f6b" event={"ID":"89b150b3-cdf8-4bc2-80db-03a6c7748aaa","Type":"ContainerStarted","Data":"ba9e1d2017d964237943de8fad99745e76d728fcb91d16efed7aba007a10fc47"}
Jul 15 17:10:20.712008 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.711987    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89a262f5-ce88-4325-9cca-ab0eaf765cac-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-7f7qf\" (UID: \"89a262f5-ce88-4325-9cca-ab0eaf765cac\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-7f7qf"
Jul 15 17:10:20.712074 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.712029    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsmbj\" (UniqueName: \"kubernetes.io/projected/89a262f5-ce88-4325-9cca-ab0eaf765cac-kube-api-access-fsmbj\") pod \"payload-pre-processing-74584bb9f8-7f7qf\" (UID: \"89a262f5-ce88-4325-9cca-ab0eaf765cac\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-7f7qf"
Jul 15 17:10:20.712074 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.712063    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/353226b7-3a58-48c2-9f48-610b86e44f37-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-ngw49\" (UID: \"353226b7-3a58-48c2-9f48-610b86e44f37\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-ngw49"
Jul 15 17:10:20.712139 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.712101    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttf6q\" (UniqueName: \"kubernetes.io/projected/353226b7-3a58-48c2-9f48-610b86e44f37-kube-api-access-ttf6q\") pod \"payload-processing-f9b8d9b9b-ngw49\" (UID: \"353226b7-3a58-48c2-9f48-610b86e44f37\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-ngw49"
Jul 15 17:10:20.813357 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.813317    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89a262f5-ce88-4325-9cca-ab0eaf765cac-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-7f7qf\" (UID: \"89a262f5-ce88-4325-9cca-ab0eaf765cac\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-7f7qf"
Jul 15 17:10:20.813550 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.813383    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsmbj\" (UniqueName: \"kubernetes.io/projected/89a262f5-ce88-4325-9cca-ab0eaf765cac-kube-api-access-fsmbj\") pod \"payload-pre-processing-74584bb9f8-7f7qf\" (UID: \"89a262f5-ce88-4325-9cca-ab0eaf765cac\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-7f7qf"
Jul 15 17:10:20.813550 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.813419    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/353226b7-3a58-48c2-9f48-610b86e44f37-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-ngw49\" (UID: \"353226b7-3a58-48c2-9f48-610b86e44f37\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-ngw49"
Jul 15 17:10:20.813550 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.813487    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttf6q\" (UniqueName: \"kubernetes.io/projected/353226b7-3a58-48c2-9f48-610b86e44f37-kube-api-access-ttf6q\") pod \"payload-processing-f9b8d9b9b-ngw49\" (UID: \"353226b7-3a58-48c2-9f48-610b86e44f37\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-ngw49"
Jul 15 17:10:20.814158 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.814140    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89a262f5-ce88-4325-9cca-ab0eaf765cac-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-7f7qf\" (UID: \"89a262f5-ce88-4325-9cca-ab0eaf765cac\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-7f7qf"
Jul 15 17:10:20.814403 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.814382    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/353226b7-3a58-48c2-9f48-610b86e44f37-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-ngw49\" (UID: \"353226b7-3a58-48c2-9f48-610b86e44f37\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-ngw49"
Jul 15 17:10:20.821888 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.821865    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttf6q\" (UniqueName: \"kubernetes.io/projected/353226b7-3a58-48c2-9f48-610b86e44f37-kube-api-access-ttf6q\") pod \"payload-processing-f9b8d9b9b-ngw49\" (UID: \"353226b7-3a58-48c2-9f48-610b86e44f37\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-ngw49"
Jul 15 17:10:20.822076 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.822053    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsmbj\" (UniqueName: \"kubernetes.io/projected/89a262f5-ce88-4325-9cca-ab0eaf765cac-kube-api-access-fsmbj\") pod \"payload-pre-processing-74584bb9f8-7f7qf\" (UID: \"89a262f5-ce88-4325-9cca-ab0eaf765cac\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-7f7qf"
Jul 15 17:10:20.964487 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.964414    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-ngw49"
Jul 15 17:10:20.971850 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:20.971828    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-7f7qf"
Jul 15 17:10:21.185095 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:21.185063    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-ngw49"]
Jul 15 17:10:21.185625 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:10:21.185595    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod353226b7_3a58_48c2_9f48_610b86e44f37.slice/crio-3ba2084f28ef358489eb9e0e24fb6dc4f4608b6386968792d9b51bfbf98b6fd1 WatchSource:0}: Error finding container 3ba2084f28ef358489eb9e0e24fb6dc4f4608b6386968792d9b51bfbf98b6fd1: Status 404 returned error can't find the container with id 3ba2084f28ef358489eb9e0e24fb6dc4f4608b6386968792d9b51bfbf98b6fd1
Jul 15 17:10:21.203791 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:21.203769    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-7f7qf"]
Jul 15 17:10:21.205875 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:10:21.205850    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89a262f5_ce88_4325_9cca_ab0eaf765cac.slice/crio-d6d45ef7f757288225303b9039f435f431743941b79ec3b1e46c554c386d8b4c WatchSource:0}: Error finding container d6d45ef7f757288225303b9039f435f431743941b79ec3b1e46c554c386d8b4c: Status 404 returned error can't find the container with id d6d45ef7f757288225303b9039f435f431743941b79ec3b1e46c554c386d8b4c
Jul 15 17:10:21.710007 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:21.709960    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-7f7qf" event={"ID":"89a262f5-ce88-4325-9cca-ab0eaf765cac","Type":"ContainerStarted","Data":"d6d45ef7f757288225303b9039f435f431743941b79ec3b1e46c554c386d8b4c"}
Jul 15 17:10:21.711928 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:21.711894    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-ngw49" event={"ID":"353226b7-3a58-48c2-9f48-610b86e44f37","Type":"ContainerStarted","Data":"3ba2084f28ef358489eb9e0e24fb6dc4f4608b6386968792d9b51bfbf98b6fd1"}
Jul 15 17:10:23.724010 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:23.723979    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-ngw49" event={"ID":"353226b7-3a58-48c2-9f48-610b86e44f37","Type":"ContainerStarted","Data":"30f24fa7a96325e159237ff9a8a309937277cf05f93741e9af5c5215460af036"}
Jul 15 17:10:23.724450 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:23.724080    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-ngw49"
Jul 15 17:10:23.725401 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:23.725373    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f869c5dd8-t4f6b" event={"ID":"89b150b3-cdf8-4bc2-80db-03a6c7748aaa","Type":"ContainerStarted","Data":"7e6c3a3d507ed7932d07c4fac949437af93bbef4eec5204dc83c31609ffab6f8"}
Jul 15 17:10:23.725518 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:23.725426    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7f869c5dd8-t4f6b"
Jul 15 17:10:23.726837 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:23.726812    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-7f7qf" event={"ID":"89a262f5-ce88-4325-9cca-ab0eaf765cac","Type":"ContainerStarted","Data":"5500b5ed466bc06cc220cdbd92eb97a8daaa04061f299003b06ff6865a4f5437"}
Jul 15 17:10:23.726946 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:23.726927    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-7f7qf"
Jul 15 17:10:23.747992 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:23.747726    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-ngw49" podStartSLOduration=1.44194037 podStartE2EDuration="3.747707249s" totalImagesPullingTime="2.305766879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:10:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:10:23.744818617 +0000 UTC m=+839.762231725" watchObservedRunningTime="2026-07-15 17:10:23.747707249 +0000 UTC m=+839.765120346"
Jul 15 17:10:23.764741 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:23.764690    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-7f7qf" podStartSLOduration=1.802330697 podStartE2EDuration="3.76467153s" totalImagesPullingTime="1.962340833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:10:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:10:23.760849627 +0000 UTC m=+839.778262724" watchObservedRunningTime="2026-07-15 17:10:23.76467153 +0000 UTC m=+839.782084627"
Jul 15 17:10:23.780537 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:23.780462    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7f869c5dd8-t4f6b" podStartSLOduration=1.162763179 podStartE2EDuration="3.780443402s" totalImagesPullingTime="2.617680223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:10:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:10:23.779486229 +0000 UTC m=+839.796899319" watchObservedRunningTime="2026-07-15 17:10:23.780443402 +0000 UTC m=+839.797856502"
Jul 15 17:10:28.751836 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:28.751800    2575 generic.go:356] "Generic (PLEG): container finished" podID="01190a7c-1f78-4e65-8582-73df15d76a5e" containerID="80027fc86c07e0ac37dd131a234dfc192fea664d5a2c5c00762354995cb4d794" exitCode=0
Jul 15 17:10:28.752221 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:28.751872    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr" event={"ID":"01190a7c-1f78-4e65-8582-73df15d76a5e","Type":"ContainerDied","Data":"80027fc86c07e0ac37dd131a234dfc192fea664d5a2c5c00762354995cb4d794"}
Jul 15 17:10:28.752268 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:28.752226    2575 scope.go:117] "RemoveContainer" containerID="80027fc86c07e0ac37dd131a234dfc192fea664d5a2c5c00762354995cb4d794"
Jul 15 17:10:28.753191 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:28.753165    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:10:28.753240 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:28.753232    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:10:28.753273 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:28.753265    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:10:29.737458 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:29.737426    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7f869c5dd8-t4f6b"
Jul 15 17:10:29.761613 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:29.761543    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr" event={"ID":"01190a7c-1f78-4e65-8582-73df15d76a5e","Type":"ContainerStarted","Data":"0bfbf2bc5cbcf22aad6511be4ddd75ee6e72fb19453b6bf8bf22784ab7e927e5"}
Jul 15 17:10:30.308151 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:30.308105    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:10:30.656690 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:30.656661    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:10:30.658353 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:30.658226    2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-nhsnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 15 17:10:30.658689 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:30.658664    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr" podUID="01190a7c-1f78-4e65-8582-73df15d76a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 15 17:10:31.657651 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:31.657613    2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-nhsnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 15 17:10:31.658146 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:31.657687    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr" podUID="01190a7c-1f78-4e65-8582-73df15d76a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 15 17:10:32.657049 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:32.657011    2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-nhsnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 15 17:10:32.657340 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:32.657309    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr" podUID="01190a7c-1f78-4e65-8582-73df15d76a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 15 17:10:33.661066 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:33.661014    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:10:33.662266 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:33.662246    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-nhsnr"
Jul 15 17:10:34.733028 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:34.732977    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-74584bb9f8-7f7qf" podUID="89a262f5-ce88-4325-9cca-ab0eaf765cac" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.43:9004: connect: connection refused"
Jul 15 17:10:34.733474 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:34.733383    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-ngw49"
Jul 15 17:10:44.732366 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:44.732335    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-7f7qf"
Jul 15 17:10:56.092734 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.092692    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"]
Jul 15 17:10:56.095463 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.095440    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:10:56.097949 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.097929    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 15 17:10:56.098218 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.098203    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-k5dwz\""
Jul 15 17:10:56.099233 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.099216    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 15 17:10:56.099334 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.099258    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 15 17:10:56.108152 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.108129    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"]
Jul 15 17:10:56.215057 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.215023    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/573d4a56-b7ee-4f30-97e7-1f985ead79b4-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm\" (UID: \"573d4a56-b7ee-4f30-97e7-1f985ead79b4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:10:56.215231 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.215075    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd5t5\" (UniqueName: \"kubernetes.io/projected/573d4a56-b7ee-4f30-97e7-1f985ead79b4-kube-api-access-wd5t5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm\" (UID: \"573d4a56-b7ee-4f30-97e7-1f985ead79b4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:10:56.215231 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.215106    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/573d4a56-b7ee-4f30-97e7-1f985ead79b4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm\" (UID: \"573d4a56-b7ee-4f30-97e7-1f985ead79b4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:10:56.215231 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.215136    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/573d4a56-b7ee-4f30-97e7-1f985ead79b4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm\" (UID: \"573d4a56-b7ee-4f30-97e7-1f985ead79b4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:10:56.215231 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.215165    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/573d4a56-b7ee-4f30-97e7-1f985ead79b4-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm\" (UID: \"573d4a56-b7ee-4f30-97e7-1f985ead79b4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:10:56.215231 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.215195    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/573d4a56-b7ee-4f30-97e7-1f985ead79b4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm\" (UID: \"573d4a56-b7ee-4f30-97e7-1f985ead79b4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:10:56.315543 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.315515    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/573d4a56-b7ee-4f30-97e7-1f985ead79b4-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm\" (UID: \"573d4a56-b7ee-4f30-97e7-1f985ead79b4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:10:56.315726 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.315589    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd5t5\" (UniqueName: \"kubernetes.io/projected/573d4a56-b7ee-4f30-97e7-1f985ead79b4-kube-api-access-wd5t5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm\" (UID: \"573d4a56-b7ee-4f30-97e7-1f985ead79b4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:10:56.315726 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.315624    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/573d4a56-b7ee-4f30-97e7-1f985ead79b4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm\" (UID: \"573d4a56-b7ee-4f30-97e7-1f985ead79b4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:10:56.315726 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.315670    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/573d4a56-b7ee-4f30-97e7-1f985ead79b4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm\" (UID: \"573d4a56-b7ee-4f30-97e7-1f985ead79b4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:10:56.315726 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.315705    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/573d4a56-b7ee-4f30-97e7-1f985ead79b4-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm\" (UID: \"573d4a56-b7ee-4f30-97e7-1f985ead79b4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:10:56.315939 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.315728    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/573d4a56-b7ee-4f30-97e7-1f985ead79b4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm\" (UID: \"573d4a56-b7ee-4f30-97e7-1f985ead79b4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:10:56.316052 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.316026    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/573d4a56-b7ee-4f30-97e7-1f985ead79b4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm\" (UID: \"573d4a56-b7ee-4f30-97e7-1f985ead79b4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:10:56.316120 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.316072    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/573d4a56-b7ee-4f30-97e7-1f985ead79b4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm\" (UID: \"573d4a56-b7ee-4f30-97e7-1f985ead79b4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:10:56.316160 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.316108    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/573d4a56-b7ee-4f30-97e7-1f985ead79b4-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm\" (UID: \"573d4a56-b7ee-4f30-97e7-1f985ead79b4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:10:56.317813 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.317788    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/573d4a56-b7ee-4f30-97e7-1f985ead79b4-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm\" (UID: \"573d4a56-b7ee-4f30-97e7-1f985ead79b4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:10:56.318059 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.318043    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/573d4a56-b7ee-4f30-97e7-1f985ead79b4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm\" (UID: \"573d4a56-b7ee-4f30-97e7-1f985ead79b4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:10:56.325598 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.325575    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd5t5\" (UniqueName: \"kubernetes.io/projected/573d4a56-b7ee-4f30-97e7-1f985ead79b4-kube-api-access-wd5t5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm\" (UID: \"573d4a56-b7ee-4f30-97e7-1f985ead79b4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:10:56.406356 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.406318    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:10:56.537548 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.537528    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"]
Jul 15 17:10:56.539970 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:10:56.539931    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod573d4a56_b7ee_4f30_97e7_1f985ead79b4.slice/crio-cfbb81bca453b12673a60660fefa4c48921509f1db4610cc7a5163049b25599c WatchSource:0}: Error finding container cfbb81bca453b12673a60660fefa4c48921509f1db4610cc7a5163049b25599c: Status 404 returned error can't find the container with id cfbb81bca453b12673a60660fefa4c48921509f1db4610cc7a5163049b25599c
Jul 15 17:10:56.894000 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:56.893962    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm" event={"ID":"573d4a56-b7ee-4f30-97e7-1f985ead79b4","Type":"ContainerStarted","Data":"cfbb81bca453b12673a60660fefa4c48921509f1db4610cc7a5163049b25599c"}
Jul 15 17:10:57.900109 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:57.900070    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm" event={"ID":"573d4a56-b7ee-4f30-97e7-1f985ead79b4","Type":"ContainerStarted","Data":"8aa0c88c74260dbc898488da841c9035d16ea7eae62bf1adced094887f2601f7"}
Jul 15 17:10:57.919512 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:10:57.919460    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm" podStartSLOduration=0.664689324 podStartE2EDuration="1.919443832s" totalImagesPullingTime="1.254754508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:10:57.91748807 +0000 UTC m=+873.934901167" watchObservedRunningTime="2026-07-15 17:10:57.919443832 +0000 UTC m=+873.936856929"
Jul 15 17:11:06.407228 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:06.407190    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:11:06.407776 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:06.407298    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:11:06.419479 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:06.419451    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:11:06.955048 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:06.955021    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm"
Jul 15 17:11:12.797820 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:12.797784    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"]
Jul 15 17:11:12.800522 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:12.800500    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:12.803395 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:12.803375    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 15 17:11:12.809414 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:12.809393    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"]
Jul 15 17:11:12.856835 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:12.856804    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9933c89-3226-4ce2-9900-675eaeccc0ed-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbl76\" (UID: \"e9933c89-3226-4ce2-9900-675eaeccc0ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:12.856967 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:12.856907    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9933c89-3226-4ce2-9900-675eaeccc0ed-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbl76\" (UID: \"e9933c89-3226-4ce2-9900-675eaeccc0ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:12.856967 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:12.856939    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9933c89-3226-4ce2-9900-675eaeccc0ed-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbl76\" (UID: \"e9933c89-3226-4ce2-9900-675eaeccc0ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:12.857063 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:12.856977    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhm55\" (UniqueName: \"kubernetes.io/projected/e9933c89-3226-4ce2-9900-675eaeccc0ed-kube-api-access-fhm55\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbl76\" (UID: \"e9933c89-3226-4ce2-9900-675eaeccc0ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:12.857063 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:12.857010    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9933c89-3226-4ce2-9900-675eaeccc0ed-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbl76\" (UID: \"e9933c89-3226-4ce2-9900-675eaeccc0ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:12.857171 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:12.857070    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9933c89-3226-4ce2-9900-675eaeccc0ed-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbl76\" (UID: \"e9933c89-3226-4ce2-9900-675eaeccc0ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:12.958284 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:12.958253    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9933c89-3226-4ce2-9900-675eaeccc0ed-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbl76\" (UID: \"e9933c89-3226-4ce2-9900-675eaeccc0ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:12.958455 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:12.958311    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9933c89-3226-4ce2-9900-675eaeccc0ed-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbl76\" (UID: \"e9933c89-3226-4ce2-9900-675eaeccc0ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:12.958455 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:12.958331    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9933c89-3226-4ce2-9900-675eaeccc0ed-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbl76\" (UID: \"e9933c89-3226-4ce2-9900-675eaeccc0ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:12.958455 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:12.958349    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhm55\" (UniqueName: \"kubernetes.io/projected/e9933c89-3226-4ce2-9900-675eaeccc0ed-kube-api-access-fhm55\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbl76\" (UID: \"e9933c89-3226-4ce2-9900-675eaeccc0ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:12.958661 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:12.958444    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9933c89-3226-4ce2-9900-675eaeccc0ed-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbl76\" (UID: \"e9933c89-3226-4ce2-9900-675eaeccc0ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:12.958661 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:12.958504    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9933c89-3226-4ce2-9900-675eaeccc0ed-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbl76\" (UID: \"e9933c89-3226-4ce2-9900-675eaeccc0ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:12.958752 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:12.958722    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9933c89-3226-4ce2-9900-675eaeccc0ed-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbl76\" (UID: \"e9933c89-3226-4ce2-9900-675eaeccc0ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:12.958752 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:12.958747    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9933c89-3226-4ce2-9900-675eaeccc0ed-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbl76\" (UID: \"e9933c89-3226-4ce2-9900-675eaeccc0ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:12.958970 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:12.958951    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9933c89-3226-4ce2-9900-675eaeccc0ed-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbl76\" (UID: \"e9933c89-3226-4ce2-9900-675eaeccc0ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:12.960658 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:12.960637    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9933c89-3226-4ce2-9900-675eaeccc0ed-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbl76\" (UID: \"e9933c89-3226-4ce2-9900-675eaeccc0ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:12.960978 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:12.960947    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9933c89-3226-4ce2-9900-675eaeccc0ed-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbl76\" (UID: \"e9933c89-3226-4ce2-9900-675eaeccc0ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:12.966941 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:12.966919    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhm55\" (UniqueName: \"kubernetes.io/projected/e9933c89-3226-4ce2-9900-675eaeccc0ed-kube-api-access-fhm55\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbl76\" (UID: \"e9933c89-3226-4ce2-9900-675eaeccc0ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:13.134261 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:13.134225    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:13.266140 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:13.266108    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"]
Jul 15 17:11:13.268612 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:11:13.268586    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9933c89_3226_4ce2_9900_675eaeccc0ed.slice/crio-0af728f6290807dae4e1406f845301f6b4c27fd1bfaec0d34c5081911d9168ce WatchSource:0}: Error finding container 0af728f6290807dae4e1406f845301f6b4c27fd1bfaec0d34c5081911d9168ce: Status 404 returned error can't find the container with id 0af728f6290807dae4e1406f845301f6b4c27fd1bfaec0d34c5081911d9168ce
Jul 15 17:11:13.975440 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:13.975400    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76" event={"ID":"e9933c89-3226-4ce2-9900-675eaeccc0ed","Type":"ContainerStarted","Data":"f6305c108b896f702c51ad30c452cebea1dd3acd8ed518626f4a8e18863b9886"}
Jul 15 17:11:13.975818 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:13.975469    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76" event={"ID":"e9933c89-3226-4ce2-9900-675eaeccc0ed","Type":"ContainerStarted","Data":"0af728f6290807dae4e1406f845301f6b4c27fd1bfaec0d34c5081911d9168ce"}
Jul 15 17:11:13.993179 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:13.993127    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76" podStartSLOduration=1.763117232 podStartE2EDuration="1.99311361s" totalImagesPullingTime="229.996378ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:11:13.991198663 +0000 UTC m=+890.008611759" watchObservedRunningTime="2026-07-15 17:11:13.99311361 +0000 UTC m=+890.010526732"
Jul 15 17:11:23.135327 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:23.135287    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:23.135792 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:23.135370    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:23.151700 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:23.151677    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:24.035747 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:24.035720    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbl76"
Jul 15 17:11:24.424281 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:24.424255    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 17:11:24.424745 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:24.424485    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 17:11:40.914812 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:40.914782    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"]
Jul 15 17:11:40.918905 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:40.918883    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:40.921390 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:40.921369    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 15 17:11:40.929264 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:40.929244    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"]
Jul 15 17:11:40.997786 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:40.997752    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8jl4\" (UniqueName: \"kubernetes.io/projected/d90e633a-075d-4944-958f-af94b19ffee4-kube-api-access-b8jl4\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw\" (UID: \"d90e633a-075d-4944-958f-af94b19ffee4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:40.997942 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:40.997813    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d90e633a-075d-4944-958f-af94b19ffee4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw\" (UID: \"d90e633a-075d-4944-958f-af94b19ffee4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:40.997942 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:40.997841    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d90e633a-075d-4944-958f-af94b19ffee4-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw\" (UID: \"d90e633a-075d-4944-958f-af94b19ffee4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:40.997942 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:40.997860    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d90e633a-075d-4944-958f-af94b19ffee4-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw\" (UID: \"d90e633a-075d-4944-958f-af94b19ffee4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:40.997942 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:40.997877    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d90e633a-075d-4944-958f-af94b19ffee4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw\" (UID: \"d90e633a-075d-4944-958f-af94b19ffee4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:40.997942 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:40.997918    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d90e633a-075d-4944-958f-af94b19ffee4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw\" (UID: \"d90e633a-075d-4944-958f-af94b19ffee4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:41.099231 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:41.099189    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8jl4\" (UniqueName: \"kubernetes.io/projected/d90e633a-075d-4944-958f-af94b19ffee4-kube-api-access-b8jl4\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw\" (UID: \"d90e633a-075d-4944-958f-af94b19ffee4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:41.099395 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:41.099352    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d90e633a-075d-4944-958f-af94b19ffee4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw\" (UID: \"d90e633a-075d-4944-958f-af94b19ffee4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:41.099463 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:41.099395    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d90e633a-075d-4944-958f-af94b19ffee4-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw\" (UID: \"d90e633a-075d-4944-958f-af94b19ffee4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:41.099463 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:41.099418    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d90e633a-075d-4944-958f-af94b19ffee4-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw\" (UID: \"d90e633a-075d-4944-958f-af94b19ffee4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:41.099593 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:41.099511    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d90e633a-075d-4944-958f-af94b19ffee4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw\" (UID: \"d90e633a-075d-4944-958f-af94b19ffee4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:41.099648 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:41.099621    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d90e633a-075d-4944-958f-af94b19ffee4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw\" (UID: \"d90e633a-075d-4944-958f-af94b19ffee4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:41.099856 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:41.099835    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d90e633a-075d-4944-958f-af94b19ffee4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw\" (UID: \"d90e633a-075d-4944-958f-af94b19ffee4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:41.099930 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:41.099838    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d90e633a-075d-4944-958f-af94b19ffee4-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw\" (UID: \"d90e633a-075d-4944-958f-af94b19ffee4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:41.099930 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:41.099903    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d90e633a-075d-4944-958f-af94b19ffee4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw\" (UID: \"d90e633a-075d-4944-958f-af94b19ffee4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:41.101530 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:41.101511    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d90e633a-075d-4944-958f-af94b19ffee4-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw\" (UID: \"d90e633a-075d-4944-958f-af94b19ffee4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:41.101792 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:41.101774    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d90e633a-075d-4944-958f-af94b19ffee4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw\" (UID: \"d90e633a-075d-4944-958f-af94b19ffee4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:41.107474 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:41.107449    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8jl4\" (UniqueName: \"kubernetes.io/projected/d90e633a-075d-4944-958f-af94b19ffee4-kube-api-access-b8jl4\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw\" (UID: \"d90e633a-075d-4944-958f-af94b19ffee4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:41.229855 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:41.229771    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:41.362298 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:41.362274    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"]
Jul 15 17:11:41.364454 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:11:41.364425    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd90e633a_075d_4944_958f_af94b19ffee4.slice/crio-dd48706413fd81c217d5bd838ea4278b9850d99f8d5fb07128273236a880aec5 WatchSource:0}: Error finding container dd48706413fd81c217d5bd838ea4278b9850d99f8d5fb07128273236a880aec5: Status 404 returned error can't find the container with id dd48706413fd81c217d5bd838ea4278b9850d99f8d5fb07128273236a880aec5
Jul 15 17:11:42.119152 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:42.119118    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw" event={"ID":"d90e633a-075d-4944-958f-af94b19ffee4","Type":"ContainerStarted","Data":"7f349885a2cd559348e91d31487e8bdb5bbdcc89624578d010d804e1335f103f"}
Jul 15 17:11:42.119638 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:42.119157    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw" event={"ID":"d90e633a-075d-4944-958f-af94b19ffee4","Type":"ContainerStarted","Data":"dd48706413fd81c217d5bd838ea4278b9850d99f8d5fb07128273236a880aec5"}
Jul 15 17:11:42.138335 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:42.138288    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw" podStartSLOduration=1.956296514 podStartE2EDuration="2.138273424s" totalImagesPullingTime="181.97691ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:11:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:11:42.135723466 +0000 UTC m=+918.153136582" watchObservedRunningTime="2026-07-15 17:11:42.138273424 +0000 UTC m=+918.155686520"
Jul 15 17:11:51.230237 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:51.230189    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:51.230237 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:51.230249    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:51.242915 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:51.242891    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:11:52.178594 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:11:52.178551    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw"
Jul 15 17:13:47.220477 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:47.220442    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b8d7f4fcb-pb8rg"]
Jul 15 17:13:47.221041 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:47.220652    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6b8d7f4fcb-pb8rg" podUID="9d6d032c-dc68-4402-9151-d6a18d905d02" containerName="manager" containerID="cri-o://8e65f384a9e01dbd734884c75ca56e90c26c5ed18e8c1b4d3b07ba12a71d1d53" gracePeriod=30
Jul 15 17:13:47.474674 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:47.474618    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b8d7f4fcb-pb8rg"
Jul 15 17:13:47.581187 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:47.581157    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d6d032c-dc68-4402-9151-d6a18d905d02-webhook-cert\") pod \"9d6d032c-dc68-4402-9151-d6a18d905d02\" (UID: \"9d6d032c-dc68-4402-9151-d6a18d905d02\") "
Jul 15 17:13:47.581341 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:47.581209    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dl7m4\" (UniqueName: \"kubernetes.io/projected/9d6d032c-dc68-4402-9151-d6a18d905d02-kube-api-access-dl7m4\") pod \"9d6d032c-dc68-4402-9151-d6a18d905d02\" (UID: \"9d6d032c-dc68-4402-9151-d6a18d905d02\") "
Jul 15 17:13:47.583165 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:47.583136    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d6d032c-dc68-4402-9151-d6a18d905d02-kube-api-access-dl7m4" (OuterVolumeSpecName: "kube-api-access-dl7m4") pod "9d6d032c-dc68-4402-9151-d6a18d905d02" (UID: "9d6d032c-dc68-4402-9151-d6a18d905d02"). InnerVolumeSpecName "kube-api-access-dl7m4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:13:47.583289 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:47.583169    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d6d032c-dc68-4402-9151-d6a18d905d02-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9d6d032c-dc68-4402-9151-d6a18d905d02" (UID: "9d6d032c-dc68-4402-9151-d6a18d905d02"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:13:47.682839 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:47.682802    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dl7m4\" (UniqueName: \"kubernetes.io/projected/9d6d032c-dc68-4402-9151-d6a18d905d02-kube-api-access-dl7m4\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:13:47.682839 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:47.682833    2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d6d032c-dc68-4402-9151-d6a18d905d02-webhook-cert\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:13:47.711748 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:47.711709    2575 generic.go:356] "Generic (PLEG): container finished" podID="9d6d032c-dc68-4402-9151-d6a18d905d02" containerID="8e65f384a9e01dbd734884c75ca56e90c26c5ed18e8c1b4d3b07ba12a71d1d53" exitCode=0
Jul 15 17:13:47.711893 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:47.711765    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b8d7f4fcb-pb8rg"
Jul 15 17:13:47.711893 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:47.711769    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b8d7f4fcb-pb8rg" event={"ID":"9d6d032c-dc68-4402-9151-d6a18d905d02","Type":"ContainerDied","Data":"8e65f384a9e01dbd734884c75ca56e90c26c5ed18e8c1b4d3b07ba12a71d1d53"}
Jul 15 17:13:47.711893 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:47.711845    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b8d7f4fcb-pb8rg" event={"ID":"9d6d032c-dc68-4402-9151-d6a18d905d02","Type":"ContainerDied","Data":"01a95c8aad67d70d375ba711b6e6134f5f031761e56de0bbfc63b1ed2ba77136"}
Jul 15 17:13:47.711893 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:47.711868    2575 scope.go:117] "RemoveContainer" containerID="8e65f384a9e01dbd734884c75ca56e90c26c5ed18e8c1b4d3b07ba12a71d1d53"
Jul 15 17:13:47.720725 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:47.720703    2575 scope.go:117] "RemoveContainer" containerID="8e65f384a9e01dbd734884c75ca56e90c26c5ed18e8c1b4d3b07ba12a71d1d53"
Jul 15 17:13:47.721004 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:13:47.720985    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e65f384a9e01dbd734884c75ca56e90c26c5ed18e8c1b4d3b07ba12a71d1d53\": container with ID starting with 8e65f384a9e01dbd734884c75ca56e90c26c5ed18e8c1b4d3b07ba12a71d1d53 not found: ID does not exist" containerID="8e65f384a9e01dbd734884c75ca56e90c26c5ed18e8c1b4d3b07ba12a71d1d53"
Jul 15 17:13:47.721072 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:47.721011    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e65f384a9e01dbd734884c75ca56e90c26c5ed18e8c1b4d3b07ba12a71d1d53"} err="failed to get container status \"8e65f384a9e01dbd734884c75ca56e90c26c5ed18e8c1b4d3b07ba12a71d1d53\": rpc error: code = NotFound desc = could not find container \"8e65f384a9e01dbd734884c75ca56e90c26c5ed18e8c1b4d3b07ba12a71d1d53\": container with ID starting with 8e65f384a9e01dbd734884c75ca56e90c26c5ed18e8c1b4d3b07ba12a71d1d53 not found: ID does not exist"
Jul 15 17:13:47.732682 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:47.732630    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b8d7f4fcb-pb8rg"]
Jul 15 17:13:47.736218 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:47.736196    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6b8d7f4fcb-pb8rg"]
Jul 15 17:13:48.526188 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:48.526156    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d6d032c-dc68-4402-9151-d6a18d905d02" path="/var/lib/kubelet/pods/9d6d032c-dc68-4402-9151-d6a18d905d02/volumes"
Jul 15 17:13:48.526666 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:48.526505    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b8d7f4fcb-84rmq"]
Jul 15 17:13:48.526937 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:48.526918    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d6d032c-dc68-4402-9151-d6a18d905d02" containerName="manager"
Jul 15 17:13:48.527017 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:48.526939    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d6d032c-dc68-4402-9151-d6a18d905d02" containerName="manager"
Jul 15 17:13:48.527072 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:48.527035    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d6d032c-dc68-4402-9151-d6a18d905d02" containerName="manager"
Jul 15 17:13:48.529993 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:48.529972    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b8d7f4fcb-84rmq"
Jul 15 17:13:48.532875 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:48.532852    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b8d7f4fcb-84rmq"]
Jul 15 17:13:48.533500 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:48.533484    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 15 17:13:48.533755 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:48.533740    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 15 17:13:48.533841 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:48.533795    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nr2cc\""
Jul 15 17:13:48.590786 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:48.590748    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwk2t\" (UniqueName: \"kubernetes.io/projected/39851bee-faca-4dea-843c-4baa4695efc3-kube-api-access-hwk2t\") pod \"maas-controller-6b8d7f4fcb-84rmq\" (UID: \"39851bee-faca-4dea-843c-4baa4695efc3\") " pod="opendatahub/maas-controller-6b8d7f4fcb-84rmq"
Jul 15 17:13:48.590956 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:48.590820    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/39851bee-faca-4dea-843c-4baa4695efc3-webhook-cert\") pod \"maas-controller-6b8d7f4fcb-84rmq\" (UID: \"39851bee-faca-4dea-843c-4baa4695efc3\") " pod="opendatahub/maas-controller-6b8d7f4fcb-84rmq"
Jul 15 17:13:48.691904 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:48.691871    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwk2t\" (UniqueName: \"kubernetes.io/projected/39851bee-faca-4dea-843c-4baa4695efc3-kube-api-access-hwk2t\") pod \"maas-controller-6b8d7f4fcb-84rmq\" (UID: \"39851bee-faca-4dea-843c-4baa4695efc3\") " pod="opendatahub/maas-controller-6b8d7f4fcb-84rmq"
Jul 15 17:13:48.692079 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:48.691931    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/39851bee-faca-4dea-843c-4baa4695efc3-webhook-cert\") pod \"maas-controller-6b8d7f4fcb-84rmq\" (UID: \"39851bee-faca-4dea-843c-4baa4695efc3\") " pod="opendatahub/maas-controller-6b8d7f4fcb-84rmq"
Jul 15 17:13:48.694229 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:48.694210    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/39851bee-faca-4dea-843c-4baa4695efc3-webhook-cert\") pod \"maas-controller-6b8d7f4fcb-84rmq\" (UID: \"39851bee-faca-4dea-843c-4baa4695efc3\") " pod="opendatahub/maas-controller-6b8d7f4fcb-84rmq"
Jul 15 17:13:48.700932 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:48.700907    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwk2t\" (UniqueName: \"kubernetes.io/projected/39851bee-faca-4dea-843c-4baa4695efc3-kube-api-access-hwk2t\") pod \"maas-controller-6b8d7f4fcb-84rmq\" (UID: \"39851bee-faca-4dea-843c-4baa4695efc3\") " pod="opendatahub/maas-controller-6b8d7f4fcb-84rmq"
Jul 15 17:13:48.841441 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:48.841345    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b8d7f4fcb-84rmq"
Jul 15 17:13:48.977686 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:48.977653    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b8d7f4fcb-84rmq"]
Jul 15 17:13:48.981462 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:13:48.981433    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39851bee_faca_4dea_843c_4baa4695efc3.slice/crio-f1afe43fdc1cfd7924bd61f1f40202f09d478db4177f70f514178931a6b48aad WatchSource:0}: Error finding container f1afe43fdc1cfd7924bd61f1f40202f09d478db4177f70f514178931a6b48aad: Status 404 returned error can't find the container with id f1afe43fdc1cfd7924bd61f1f40202f09d478db4177f70f514178931a6b48aad
Jul 15 17:13:48.982924 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:48.982903    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 17:13:49.725941 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:49.725860    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b8d7f4fcb-84rmq" event={"ID":"39851bee-faca-4dea-843c-4baa4695efc3","Type":"ContainerStarted","Data":"c618d6b3c8b46ed8925596d8fca68f2e8dd74b081d82c8160fe67e45af0e194a"}
Jul 15 17:13:49.725941 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:49.725911    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b8d7f4fcb-84rmq" event={"ID":"39851bee-faca-4dea-843c-4baa4695efc3","Type":"ContainerStarted","Data":"f1afe43fdc1cfd7924bd61f1f40202f09d478db4177f70f514178931a6b48aad"}
Jul 15 17:13:49.726333 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:49.725961    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b8d7f4fcb-84rmq"
Jul 15 17:13:49.743922 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:13:49.743874    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6b8d7f4fcb-84rmq" podStartSLOduration=1.261358269 podStartE2EDuration="1.743859202s" totalImagesPullingTime="482.500933ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:13:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:13:49.742734376 +0000 UTC m=+1045.760147523" watchObservedRunningTime="2026-07-15 17:13:49.743859202 +0000 UTC m=+1045.761272298"
Jul 15 17:14:00.732698 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:14:00.732663    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b8d7f4fcb-84rmq"
Jul 15 17:16:24.451006 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:16:24.450978    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 17:16:24.453130 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:16:24.453110    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 17:21:24.478643 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:21:24.478510    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 17:21:24.485213 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:21:24.485181    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 17:26:12.733979 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.733937    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"]
Jul 15 17:26:12.738092 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.738068    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.740695 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.740674    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-82b50f4c-openshift-default-dockercfg-7n2hv\""
Jul 15 17:26:12.756407 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.756381    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"]
Jul 15 17:26:12.783378 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.783347    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-workload-certs\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.783378 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.783379    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db5ph\" (UniqueName: \"kubernetes.io/projected/5924f3e6-5db1-44bb-a965-39afb729ecaa-kube-api-access-db5ph\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.783588 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.783401    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5924f3e6-5db1-44bb-a965-39afb729ecaa-istiod-ca-cert\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.783588 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.783460    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-podinfo\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.783588 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.783509    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-workload-socket\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.783588 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.783528    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-credential-socket\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.783588 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.783546    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-data\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.783778 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.783620    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-envoy\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.783778 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.783649    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-token\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.884618 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.884550    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-envoy\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.884795 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.884635    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-token\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.884795 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.884662    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-workload-certs\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.884795 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.884680    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db5ph\" (UniqueName: \"kubernetes.io/projected/5924f3e6-5db1-44bb-a965-39afb729ecaa-kube-api-access-db5ph\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.884795 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.884699    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5924f3e6-5db1-44bb-a965-39afb729ecaa-istiod-ca-cert\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.884795 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.884725    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-podinfo\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.884795 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.884777    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-workload-socket\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.885110 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.884804    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-credential-socket\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.885110 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.884827    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-data\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.885110 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.885050    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-workload-certs\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.885271 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.885186    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-credential-socket\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.885271 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.885240    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-data\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.885353 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.885326    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-workload-socket\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.885488 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.885468    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5924f3e6-5db1-44bb-a965-39afb729ecaa-istiod-ca-cert\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.886861 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.886832    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-envoy\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.887326 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.887306    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-podinfo\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.892792 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.892761    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-token\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:12.892949 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:12.892929    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-db5ph\" (UniqueName: \"kubernetes.io/projected/5924f3e6-5db1-44bb-a965-39afb729ecaa-kube-api-access-db5ph\") pod \"e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") " pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:13.051221 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:13.051139    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:13.197465 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:13.197442    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"]
Jul 15 17:26:13.199789 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:26:13.199758    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5924f3e6_5db1_44bb_a965_39afb729ecaa.slice/crio-f2471aa98af0d9fe77d2cebcc480d7f3ae2f0d8f95e0b97091b161adffc17b8d WatchSource:0}: Error finding container f2471aa98af0d9fe77d2cebcc480d7f3ae2f0d8f95e0b97091b161adffc17b8d: Status 404 returned error can't find the container with id f2471aa98af0d9fe77d2cebcc480d7f3ae2f0d8f95e0b97091b161adffc17b8d
Jul 15 17:26:13.201783 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:13.201753    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:26:13.201894 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:13.201814    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:26:13.201894 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:13.201842    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:26:13.260787 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:13.260761    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2" event={"ID":"5924f3e6-5db1-44bb-a965-39afb729ecaa","Type":"ContainerStarted","Data":"f2471aa98af0d9fe77d2cebcc480d7f3ae2f0d8f95e0b97091b161adffc17b8d"}
Jul 15 17:26:14.267690 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:14.267659    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2" event={"ID":"5924f3e6-5db1-44bb-a965-39afb729ecaa","Type":"ContainerStarted","Data":"1e4c3577c4f5a3d142d234110ba2e03b96218b3688683e3641b1a9242efa3db3"}
Jul 15 17:26:14.305977 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:14.305930    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2" podStartSLOduration=2.30591249 podStartE2EDuration="2.30591249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:26:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:26:14.305041448 +0000 UTC m=+1790.322454544" watchObservedRunningTime="2026-07-15 17:26:14.30591249 +0000 UTC m=+1790.323325585"
Jul 15 17:26:15.052116 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:15.052082    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:15.057125 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:15.057096    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:15.272856 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:15.272828    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:15.274051 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:15.274029    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:24.512687 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:24.512575    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 17:26:24.518036 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:24.516240    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 17:26:29.339481 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:29.339453    2575 generic.go:356] "Generic (PLEG): container finished" podID="5924f3e6-5db1-44bb-a965-39afb729ecaa" containerID="1e4c3577c4f5a3d142d234110ba2e03b96218b3688683e3641b1a9242efa3db3" exitCode=0
Jul 15 17:26:29.339871 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:29.339510    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2" event={"ID":"5924f3e6-5db1-44bb-a965-39afb729ecaa","Type":"ContainerDied","Data":"1e4c3577c4f5a3d142d234110ba2e03b96218b3688683e3641b1a9242efa3db3"}
Jul 15 17:26:29.339936 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:29.339918    2575 scope.go:117] "RemoveContainer" containerID="1e4c3577c4f5a3d142d234110ba2e03b96218b3688683e3641b1a9242efa3db3"
Jul 15 17:26:29.340888 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:29.340864    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:26:29.340941 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:29.340926    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:26:29.340974 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:29.340959    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:26:30.273227 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:30.273186    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:30.349582 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:30.349519    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2" event={"ID":"5924f3e6-5db1-44bb-a965-39afb729ecaa","Type":"ContainerStarted","Data":"281bae6a224450b793bb835744ee8f9daf1c4faad7044cdae8d5272e6cbbb4ae"}
Jul 15 17:26:31.051474 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:31.051431    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:31.053244 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:31.053199    2575 patch_prober.go:28] interesting pod/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 15 17:26:31.053392 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:31.053298    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2" podUID="5924f3e6-5db1-44bb-a965-39afb729ecaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 15 17:26:32.051598 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:32.051542    2575 patch_prober.go:28] interesting pod/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 15 17:26:32.052029 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:32.051658    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2" podUID="5924f3e6-5db1-44bb-a965-39afb729ecaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 15 17:26:33.051851 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:33.051815    2575 patch_prober.go:28] interesting pod/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 15 17:26:33.052447 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:33.052413    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2" podUID="5924f3e6-5db1-44bb-a965-39afb729ecaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 15 17:26:34.052342 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:34.051841    2575 patch_prober.go:28] interesting pod/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 15 17:26:34.052342 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:34.051946    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2" podUID="5924f3e6-5db1-44bb-a965-39afb729ecaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 15 17:26:35.055721 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:35.055691    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:35.056155 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:35.055956    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:26:35.056707 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:26:35.056691    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:27:34.230517 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:34.230484    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"]
Jul 15 17:27:34.230935 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:34.230696    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2" podUID="5924f3e6-5db1-44bb-a965-39afb729ecaa" containerName="istio-proxy" containerID="cri-o://281bae6a224450b793bb835744ee8f9daf1c4faad7044cdae8d5272e6cbbb4ae" gracePeriod=30
Jul 15 17:27:35.058657 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:35.058623    2575 patch_prober.go:28] interesting pod/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 17:27:35.058822 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:35.058677    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2" podUID="5924f3e6-5db1-44bb-a965-39afb729ecaa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 17:27:36.400803 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:36.400762    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c"]
Jul 15 17:27:36.404195 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:36.404170    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c"
Jul 15 17:27:36.407725 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:36.407703    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-2a15ad60-serving-cert\""
Jul 15 17:27:36.419650 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:36.419629    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c"]
Jul 15 17:27:36.499474 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:36.499439    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkjbg\" (UniqueName: \"kubernetes.io/projected/10817f06-d14d-41a6-b4bd-df652a774a90-kube-api-access-tkjbg\") pod \"maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c\" (UID: \"10817f06-d14d-41a6-b4bd-df652a774a90\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c"
Jul 15 17:27:36.499662 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:36.499553    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10817f06-d14d-41a6-b4bd-df652a774a90-maas-api-tls\") pod \"maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c\" (UID: \"10817f06-d14d-41a6-b4bd-df652a774a90\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c"
Jul 15 17:27:36.499662 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:36.499630    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10817f06-d14d-41a6-b4bd-df652a774a90-openshift-service-ca\") pod \"maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c\" (UID: \"10817f06-d14d-41a6-b4bd-df652a774a90\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c"
Jul 15 17:27:36.600329 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:36.600297    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkjbg\" (UniqueName: \"kubernetes.io/projected/10817f06-d14d-41a6-b4bd-df652a774a90-kube-api-access-tkjbg\") pod \"maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c\" (UID: \"10817f06-d14d-41a6-b4bd-df652a774a90\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c"
Jul 15 17:27:36.600484 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:36.600368    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10817f06-d14d-41a6-b4bd-df652a774a90-maas-api-tls\") pod \"maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c\" (UID: \"10817f06-d14d-41a6-b4bd-df652a774a90\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c"
Jul 15 17:27:36.600484 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:36.600402    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10817f06-d14d-41a6-b4bd-df652a774a90-openshift-service-ca\") pod \"maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c\" (UID: \"10817f06-d14d-41a6-b4bd-df652a774a90\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c"
Jul 15 17:27:36.600611 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:27:36.600500    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-serving-cert: secret "maas-api-e2e-migrate-2a15ad60-serving-cert" not found
Jul 15 17:27:36.600611 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:27:36.600579    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10817f06-d14d-41a6-b4bd-df652a774a90-maas-api-tls podName:10817f06-d14d-41a6-b4bd-df652a774a90 nodeName:}" failed. No retries permitted until 2026-07-15 17:27:37.100542571 +0000 UTC m=+1873.117955645 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/10817f06-d14d-41a6-b4bd-df652a774a90-maas-api-tls") pod "maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c" (UID: "10817f06-d14d-41a6-b4bd-df652a774a90") : secret "maas-api-e2e-migrate-2a15ad60-serving-cert" not found
Jul 15 17:27:36.600969 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:36.600953    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10817f06-d14d-41a6-b4bd-df652a774a90-openshift-service-ca\") pod \"maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c\" (UID: \"10817f06-d14d-41a6-b4bd-df652a774a90\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c"
Jul 15 17:27:36.610889 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:36.610860    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkjbg\" (UniqueName: \"kubernetes.io/projected/10817f06-d14d-41a6-b4bd-df652a774a90-kube-api-access-tkjbg\") pod \"maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c\" (UID: \"10817f06-d14d-41a6-b4bd-df652a774a90\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c"
Jul 15 17:27:37.105584 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:37.105523    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10817f06-d14d-41a6-b4bd-df652a774a90-maas-api-tls\") pod \"maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c\" (UID: \"10817f06-d14d-41a6-b4bd-df652a774a90\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c"
Jul 15 17:27:37.107912 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:37.107894    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10817f06-d14d-41a6-b4bd-df652a774a90-maas-api-tls\") pod \"maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c\" (UID: \"10817f06-d14d-41a6-b4bd-df652a774a90\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c"
Jul 15 17:27:37.314323 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:37.314285    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c"
Jul 15 17:27:37.656813 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:37.656788    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c"]
Jul 15 17:27:37.659627 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:27:37.659602    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10817f06_d14d_41a6_b4bd_df652a774a90.slice/crio-e75fa3951478c11d774350545deec94a41f70d874fb3d2a84b7f5312106ab6eb WatchSource:0}: Error finding container e75fa3951478c11d774350545deec94a41f70d874fb3d2a84b7f5312106ab6eb: Status 404 returned error can't find the container with id e75fa3951478c11d774350545deec94a41f70d874fb3d2a84b7f5312106ab6eb
Jul 15 17:27:37.661449 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:37.661430    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 17:27:37.679373 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:37.679351    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c" event={"ID":"10817f06-d14d-41a6-b4bd-df652a774a90","Type":"ContainerStarted","Data":"e75fa3951478c11d774350545deec94a41f70d874fb3d2a84b7f5312106ab6eb"}
Jul 15 17:27:38.686047 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:38.686000    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c" event={"ID":"10817f06-d14d-41a6-b4bd-df652a774a90","Type":"ContainerStarted","Data":"0a85b6bb07fa58fe32b658eab7badbd1d697af9931b9408b3c0304aa06894c58"}
Jul 15 17:27:38.686429 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:38.686103    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c"
Jul 15 17:27:38.704857 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:38.704813    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c" podStartSLOduration=2.255423513 podStartE2EDuration="2.704799609s" totalImagesPullingTime="449.376096ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:27:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:27:38.703657782 +0000 UTC m=+1874.721070880" watchObservedRunningTime="2026-07-15 17:27:38.704799609 +0000 UTC m=+1874.722212704"
Jul 15 17:27:39.484943 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.484914    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:27:39.527393 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.527320    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-envoy\") pod \"5924f3e6-5db1-44bb-a965-39afb729ecaa\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") "
Jul 15 17:27:39.527393 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.527367    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-db5ph\" (UniqueName: \"kubernetes.io/projected/5924f3e6-5db1-44bb-a965-39afb729ecaa-kube-api-access-db5ph\") pod \"5924f3e6-5db1-44bb-a965-39afb729ecaa\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") "
Jul 15 17:27:39.527952 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.527406    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-workload-certs\") pod \"5924f3e6-5db1-44bb-a965-39afb729ecaa\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") "
Jul 15 17:27:39.527952 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.527434    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-podinfo\") pod \"5924f3e6-5db1-44bb-a965-39afb729ecaa\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") "
Jul 15 17:27:39.527952 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.527477    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-data\") pod \"5924f3e6-5db1-44bb-a965-39afb729ecaa\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") "
Jul 15 17:27:39.527952 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.527510    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-token\") pod \"5924f3e6-5db1-44bb-a965-39afb729ecaa\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") "
Jul 15 17:27:39.527952 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.527580    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-credential-socket\") pod \"5924f3e6-5db1-44bb-a965-39afb729ecaa\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") "
Jul 15 17:27:39.527952 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.527643    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5924f3e6-5db1-44bb-a965-39afb729ecaa-istiod-ca-cert\") pod \"5924f3e6-5db1-44bb-a965-39afb729ecaa\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") "
Jul 15 17:27:39.527952 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.527673    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-workload-socket\") pod \"5924f3e6-5db1-44bb-a965-39afb729ecaa\" (UID: \"5924f3e6-5db1-44bb-a965-39afb729ecaa\") "
Jul 15 17:27:39.527952 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.527709    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5924f3e6-5db1-44bb-a965-39afb729ecaa" (UID: "5924f3e6-5db1-44bb-a965-39afb729ecaa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:27:39.528448 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.528007    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-workload-certs\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:27:39.528448 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.528006    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5924f3e6-5db1-44bb-a965-39afb729ecaa" (UID: "5924f3e6-5db1-44bb-a965-39afb729ecaa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:27:39.528448 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.528116    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5924f3e6-5db1-44bb-a965-39afb729ecaa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5924f3e6-5db1-44bb-a965-39afb729ecaa" (UID: "5924f3e6-5db1-44bb-a965-39afb729ecaa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:27:39.528448 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.528257    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5924f3e6-5db1-44bb-a965-39afb729ecaa" (UID: "5924f3e6-5db1-44bb-a965-39afb729ecaa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:27:39.528448 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.528433    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-data" (OuterVolumeSpecName: "istio-data") pod "5924f3e6-5db1-44bb-a965-39afb729ecaa" (UID: "5924f3e6-5db1-44bb-a965-39afb729ecaa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:27:39.530301 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.530272    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5924f3e6-5db1-44bb-a965-39afb729ecaa-kube-api-access-db5ph" (OuterVolumeSpecName: "kube-api-access-db5ph") pod "5924f3e6-5db1-44bb-a965-39afb729ecaa" (UID: "5924f3e6-5db1-44bb-a965-39afb729ecaa"). InnerVolumeSpecName "kube-api-access-db5ph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:27:39.530794 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.530768    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5924f3e6-5db1-44bb-a965-39afb729ecaa" (UID: "5924f3e6-5db1-44bb-a965-39afb729ecaa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:27:39.530950 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.530870    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-token" (OuterVolumeSpecName: "istio-token") pod "5924f3e6-5db1-44bb-a965-39afb729ecaa" (UID: "5924f3e6-5db1-44bb-a965-39afb729ecaa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:27:39.530950 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.530943    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5924f3e6-5db1-44bb-a965-39afb729ecaa" (UID: "5924f3e6-5db1-44bb-a965-39afb729ecaa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 17:27:39.629127 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.629101    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-data\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:27:39.629127 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.629125    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-token\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:27:39.629282 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.629134    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-credential-socket\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:27:39.629282 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.629143    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5924f3e6-5db1-44bb-a965-39afb729ecaa-istiod-ca-cert\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:27:39.629282 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.629152    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-workload-socket\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:27:39.629282 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.629160    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-envoy\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:27:39.629282 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.629168    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-db5ph\" (UniqueName: \"kubernetes.io/projected/5924f3e6-5db1-44bb-a965-39afb729ecaa-kube-api-access-db5ph\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:27:39.629282 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.629177    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5924f3e6-5db1-44bb-a965-39afb729ecaa-istio-podinfo\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:27:39.693781 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.693749    2575 generic.go:356] "Generic (PLEG): container finished" podID="5924f3e6-5db1-44bb-a965-39afb729ecaa" containerID="281bae6a224450b793bb835744ee8f9daf1c4faad7044cdae8d5272e6cbbb4ae" exitCode=0
Jul 15 17:27:39.694162 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.693796    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"
Jul 15 17:27:39.694162 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.693830    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2" event={"ID":"5924f3e6-5db1-44bb-a965-39afb729ecaa","Type":"ContainerDied","Data":"281bae6a224450b793bb835744ee8f9daf1c4faad7044cdae8d5272e6cbbb4ae"}
Jul 15 17:27:39.694162 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.693894    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2" event={"ID":"5924f3e6-5db1-44bb-a965-39afb729ecaa","Type":"ContainerDied","Data":"f2471aa98af0d9fe77d2cebcc480d7f3ae2f0d8f95e0b97091b161adffc17b8d"}
Jul 15 17:27:39.694162 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.693923    2575 scope.go:117] "RemoveContainer" containerID="281bae6a224450b793bb835744ee8f9daf1c4faad7044cdae8d5272e6cbbb4ae"
Jul 15 17:27:39.703313 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.703299    2575 scope.go:117] "RemoveContainer" containerID="1e4c3577c4f5a3d142d234110ba2e03b96218b3688683e3641b1a9242efa3db3"
Jul 15 17:27:39.710737 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.710715    2575 scope.go:117] "RemoveContainer" containerID="281bae6a224450b793bb835744ee8f9daf1c4faad7044cdae8d5272e6cbbb4ae"
Jul 15 17:27:39.711004 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:27:39.710982    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"281bae6a224450b793bb835744ee8f9daf1c4faad7044cdae8d5272e6cbbb4ae\": container with ID starting with 281bae6a224450b793bb835744ee8f9daf1c4faad7044cdae8d5272e6cbbb4ae not found: ID does not exist" containerID="281bae6a224450b793bb835744ee8f9daf1c4faad7044cdae8d5272e6cbbb4ae"
Jul 15 17:27:39.711115 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.711009    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"281bae6a224450b793bb835744ee8f9daf1c4faad7044cdae8d5272e6cbbb4ae"} err="failed to get container status \"281bae6a224450b793bb835744ee8f9daf1c4faad7044cdae8d5272e6cbbb4ae\": rpc error: code = NotFound desc = could not find container \"281bae6a224450b793bb835744ee8f9daf1c4faad7044cdae8d5272e6cbbb4ae\": container with ID starting with 281bae6a224450b793bb835744ee8f9daf1c4faad7044cdae8d5272e6cbbb4ae not found: ID does not exist"
Jul 15 17:27:39.711115 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.711024    2575 scope.go:117] "RemoveContainer" containerID="1e4c3577c4f5a3d142d234110ba2e03b96218b3688683e3641b1a9242efa3db3"
Jul 15 17:27:39.711268 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:27:39.711252    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e4c3577c4f5a3d142d234110ba2e03b96218b3688683e3641b1a9242efa3db3\": container with ID starting with 1e4c3577c4f5a3d142d234110ba2e03b96218b3688683e3641b1a9242efa3db3 not found: ID does not exist" containerID="1e4c3577c4f5a3d142d234110ba2e03b96218b3688683e3641b1a9242efa3db3"
Jul 15 17:27:39.711317 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.711271    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e4c3577c4f5a3d142d234110ba2e03b96218b3688683e3641b1a9242efa3db3"} err="failed to get container status \"1e4c3577c4f5a3d142d234110ba2e03b96218b3688683e3641b1a9242efa3db3\": rpc error: code = NotFound desc = could not find container \"1e4c3577c4f5a3d142d234110ba2e03b96218b3688683e3641b1a9242efa3db3\": container with ID starting with 1e4c3577c4f5a3d142d234110ba2e03b96218b3688683e3641b1a9242efa3db3 not found: ID does not exist"
Jul 15 17:27:39.717662 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.717627    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"]
Jul 15 17:27:39.721398 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:39.721372    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-82b50f4c-openshift-default-7484dd9f59-xmjt2"]
Jul 15 17:27:40.526194 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:40.526162    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5924f3e6-5db1-44bb-a965-39afb729ecaa" path="/var/lib/kubelet/pods/5924f3e6-5db1-44bb-a965-39afb729ecaa/volumes"
Jul 15 17:27:44.699802 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:44.699775    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c"
Jul 15 17:27:51.188064 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:51.187967    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c"]
Jul 15 17:27:51.188510 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:51.188205    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c" podUID="10817f06-d14d-41a6-b4bd-df652a774a90" containerName="maas-api" containerID="cri-o://0a85b6bb07fa58fe32b658eab7badbd1d697af9931b9408b3c0304aa06894c58" gracePeriod=30
Jul 15 17:27:51.437670 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:51.437650    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c"
Jul 15 17:27:51.530664 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:51.530595    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10817f06-d14d-41a6-b4bd-df652a774a90-openshift-service-ca\") pod \"10817f06-d14d-41a6-b4bd-df652a774a90\" (UID: \"10817f06-d14d-41a6-b4bd-df652a774a90\") "
Jul 15 17:27:51.530664 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:51.530629    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkjbg\" (UniqueName: \"kubernetes.io/projected/10817f06-d14d-41a6-b4bd-df652a774a90-kube-api-access-tkjbg\") pod \"10817f06-d14d-41a6-b4bd-df652a774a90\" (UID: \"10817f06-d14d-41a6-b4bd-df652a774a90\") "
Jul 15 17:27:51.530664 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:51.530647    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10817f06-d14d-41a6-b4bd-df652a774a90-maas-api-tls\") pod \"10817f06-d14d-41a6-b4bd-df652a774a90\" (UID: \"10817f06-d14d-41a6-b4bd-df652a774a90\") "
Jul 15 17:27:51.531012 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:51.530987    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10817f06-d14d-41a6-b4bd-df652a774a90-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "10817f06-d14d-41a6-b4bd-df652a774a90" (UID: "10817f06-d14d-41a6-b4bd-df652a774a90"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:27:51.532832 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:51.532812    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10817f06-d14d-41a6-b4bd-df652a774a90-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "10817f06-d14d-41a6-b4bd-df652a774a90" (UID: "10817f06-d14d-41a6-b4bd-df652a774a90"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:27:51.532900 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:51.532877    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10817f06-d14d-41a6-b4bd-df652a774a90-kube-api-access-tkjbg" (OuterVolumeSpecName: "kube-api-access-tkjbg") pod "10817f06-d14d-41a6-b4bd-df652a774a90" (UID: "10817f06-d14d-41a6-b4bd-df652a774a90"). InnerVolumeSpecName "kube-api-access-tkjbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:27:51.632097 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:51.632062    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10817f06-d14d-41a6-b4bd-df652a774a90-openshift-service-ca\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:27:51.632097 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:51.632090    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkjbg\" (UniqueName: \"kubernetes.io/projected/10817f06-d14d-41a6-b4bd-df652a774a90-kube-api-access-tkjbg\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:27:51.632097 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:51.632101    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10817f06-d14d-41a6-b4bd-df652a774a90-maas-api-tls\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:27:51.753681 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:51.753651    2575 generic.go:356] "Generic (PLEG): container finished" podID="10817f06-d14d-41a6-b4bd-df652a774a90" containerID="0a85b6bb07fa58fe32b658eab7badbd1d697af9931b9408b3c0304aa06894c58" exitCode=0
Jul 15 17:27:51.753839 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:51.753699    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c"
Jul 15 17:27:51.753839 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:51.753726    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c" event={"ID":"10817f06-d14d-41a6-b4bd-df652a774a90","Type":"ContainerDied","Data":"0a85b6bb07fa58fe32b658eab7badbd1d697af9931b9408b3c0304aa06894c58"}
Jul 15 17:27:51.753839 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:51.753769    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c" event={"ID":"10817f06-d14d-41a6-b4bd-df652a774a90","Type":"ContainerDied","Data":"e75fa3951478c11d774350545deec94a41f70d874fb3d2a84b7f5312106ab6eb"}
Jul 15 17:27:51.753839 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:51.753789    2575 scope.go:117] "RemoveContainer" containerID="0a85b6bb07fa58fe32b658eab7badbd1d697af9931b9408b3c0304aa06894c58"
Jul 15 17:27:51.765515 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:51.765501    2575 scope.go:117] "RemoveContainer" containerID="0a85b6bb07fa58fe32b658eab7badbd1d697af9931b9408b3c0304aa06894c58"
Jul 15 17:27:51.765785 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:27:51.765766    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a85b6bb07fa58fe32b658eab7badbd1d697af9931b9408b3c0304aa06894c58\": container with ID starting with 0a85b6bb07fa58fe32b658eab7badbd1d697af9931b9408b3c0304aa06894c58 not found: ID does not exist" containerID="0a85b6bb07fa58fe32b658eab7badbd1d697af9931b9408b3c0304aa06894c58"
Jul 15 17:27:51.765853 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:51.765794    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a85b6bb07fa58fe32b658eab7badbd1d697af9931b9408b3c0304aa06894c58"} err="failed to get container status \"0a85b6bb07fa58fe32b658eab7badbd1d697af9931b9408b3c0304aa06894c58\": rpc error: code = NotFound desc = could not find container \"0a85b6bb07fa58fe32b658eab7badbd1d697af9931b9408b3c0304aa06894c58\": container with ID starting with 0a85b6bb07fa58fe32b658eab7badbd1d697af9931b9408b3c0304aa06894c58 not found: ID does not exist"
Jul 15 17:27:51.778568 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:51.778538    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c"]
Jul 15 17:27:51.781962 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:51.781911    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2a15ad60-5c7756bb5d-k8w2c"]
Jul 15 17:27:52.526060 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:27:52.525985    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10817f06-d14d-41a6-b4bd-df652a774a90" path="/var/lib/kubelet/pods/10817f06-d14d-41a6-b4bd-df652a774a90/volumes"
Jul 15 17:28:06.343024 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.342987    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"]
Jul 15 17:28:06.343630 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.343572    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5924f3e6-5db1-44bb-a965-39afb729ecaa" containerName="istio-proxy"
Jul 15 17:28:06.343630 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.343592    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5924f3e6-5db1-44bb-a965-39afb729ecaa" containerName="istio-proxy"
Jul 15 17:28:06.343630 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.343613    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5924f3e6-5db1-44bb-a965-39afb729ecaa" containerName="istio-proxy"
Jul 15 17:28:06.343630 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.343620    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5924f3e6-5db1-44bb-a965-39afb729ecaa" containerName="istio-proxy"
Jul 15 17:28:06.343855 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.343634    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10817f06-d14d-41a6-b4bd-df652a774a90" containerName="maas-api"
Jul 15 17:28:06.343855 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.343643    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="10817f06-d14d-41a6-b4bd-df652a774a90" containerName="maas-api"
Jul 15 17:28:06.343855 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.343746    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5924f3e6-5db1-44bb-a965-39afb729ecaa" containerName="istio-proxy"
Jul 15 17:28:06.343855 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.343758    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5924f3e6-5db1-44bb-a965-39afb729ecaa" containerName="istio-proxy"
Jul 15 17:28:06.343855 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.343768    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="10817f06-d14d-41a6-b4bd-df652a774a90" containerName="maas-api"
Jul 15 17:28:06.347355 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.347334    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.350113 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.350088    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e7083808-openshift-default-dockercfg-rhn6s\""
Jul 15 17:28:06.366288 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.366253    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"]
Jul 15 17:28:06.366948 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.366920    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-envoy\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.367047 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.366990    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-credential-socket\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.367047 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.367041    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-workload-socket\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.367154 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.367082    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-podinfo\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.367154 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.367111    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-workload-certs\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.367154 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.367143    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-token\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.367292 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.367184    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-data\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.367292 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.367230    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a699b5cf-fb45-4d08-8daa-cebc96714c68-istiod-ca-cert\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.367292 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.367266    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cktsb\" (UniqueName: \"kubernetes.io/projected/a699b5cf-fb45-4d08-8daa-cebc96714c68-kube-api-access-cktsb\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.468078 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.468046    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-envoy\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.468250 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.468098    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-credential-socket\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.468250 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.468139    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-workload-socket\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.468250 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.468172    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-podinfo\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.468250 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.468200    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-workload-certs\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.468250 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.468224    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-token\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.468518 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.468257    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-data\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.468518 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.468301    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a699b5cf-fb45-4d08-8daa-cebc96714c68-istiod-ca-cert\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.468518 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.468333    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cktsb\" (UniqueName: \"kubernetes.io/projected/a699b5cf-fb45-4d08-8daa-cebc96714c68-kube-api-access-cktsb\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.468728 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.468678    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-workload-certs\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.468728 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.468709    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-workload-socket\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.468868 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.468844    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-data\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.468982 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.468962    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-credential-socket\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.469237 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.469215    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a699b5cf-fb45-4d08-8daa-cebc96714c68-istiod-ca-cert\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.470502 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.470481    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-envoy\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.470649 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.470631    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-podinfo\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.478184 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.478163    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-token\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.478314 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.478284    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cktsb\" (UniqueName: \"kubernetes.io/projected/a699b5cf-fb45-4d08-8daa-cebc96714c68-kube-api-access-cktsb\") pod \"e2e-ait-e7083808-openshift-default-fc8b67769-74mgb\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") " pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.659153 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.659124    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:06.792288 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.792259    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"]
Jul 15 17:28:06.793956 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:28:06.793927    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda699b5cf_fb45_4d08_8daa_cebc96714c68.slice/crio-5c01e1a5fb62448856ae78ebca6d2188bd083fc6c93781c2a5b6d1414fe0de76 WatchSource:0}: Error finding container 5c01e1a5fb62448856ae78ebca6d2188bd083fc6c93781c2a5b6d1414fe0de76: Status 404 returned error can't find the container with id 5c01e1a5fb62448856ae78ebca6d2188bd083fc6c93781c2a5b6d1414fe0de76
Jul 15 17:28:06.796006 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.795975    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:28:06.796088 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.796039    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:28:06.796088 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.796068    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:28:06.826541 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:06.826514    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb" event={"ID":"a699b5cf-fb45-4d08-8daa-cebc96714c68","Type":"ContainerStarted","Data":"5c01e1a5fb62448856ae78ebca6d2188bd083fc6c93781c2a5b6d1414fe0de76"}
Jul 15 17:28:07.628033 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:07.627999    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8"]
Jul 15 17:28:07.632153 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:07.632134    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8"
Jul 15 17:28:07.635147 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:07.635128    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e7083808-serving-cert\""
Jul 15 17:28:07.644087 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:07.644067    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8"]
Jul 15 17:28:07.679024 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:07.678998    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcrv2\" (UniqueName: \"kubernetes.io/projected/9a9ae8b3-2b6b-47ec-98e4-c13603084168-kube-api-access-jcrv2\") pod \"maas-api-e2e-ait-e7083808-66789766fb-flrh8\" (UID: \"9a9ae8b3-2b6b-47ec-98e4-c13603084168\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8"
Jul 15 17:28:07.679188 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:07.679064    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a9ae8b3-2b6b-47ec-98e4-c13603084168-maas-api-tls\") pod \"maas-api-e2e-ait-e7083808-66789766fb-flrh8\" (UID: \"9a9ae8b3-2b6b-47ec-98e4-c13603084168\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8"
Jul 15 17:28:07.679188 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:07.679146    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a9ae8b3-2b6b-47ec-98e4-c13603084168-openshift-service-ca\") pod \"maas-api-e2e-ait-e7083808-66789766fb-flrh8\" (UID: \"9a9ae8b3-2b6b-47ec-98e4-c13603084168\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8"
Jul 15 17:28:07.780196 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:07.780160    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a9ae8b3-2b6b-47ec-98e4-c13603084168-openshift-service-ca\") pod \"maas-api-e2e-ait-e7083808-66789766fb-flrh8\" (UID: \"9a9ae8b3-2b6b-47ec-98e4-c13603084168\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8"
Jul 15 17:28:07.780196 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:07.780196    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcrv2\" (UniqueName: \"kubernetes.io/projected/9a9ae8b3-2b6b-47ec-98e4-c13603084168-kube-api-access-jcrv2\") pod \"maas-api-e2e-ait-e7083808-66789766fb-flrh8\" (UID: \"9a9ae8b3-2b6b-47ec-98e4-c13603084168\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8"
Jul 15 17:28:07.780392 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:07.780237    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a9ae8b3-2b6b-47ec-98e4-c13603084168-maas-api-tls\") pod \"maas-api-e2e-ait-e7083808-66789766fb-flrh8\" (UID: \"9a9ae8b3-2b6b-47ec-98e4-c13603084168\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8"
Jul 15 17:28:07.780392 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:28:07.780340    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-serving-cert: secret "maas-api-e2e-ait-e7083808-serving-cert" not found
Jul 15 17:28:07.780392 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:28:07.780390    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a9ae8b3-2b6b-47ec-98e4-c13603084168-maas-api-tls podName:9a9ae8b3-2b6b-47ec-98e4-c13603084168 nodeName:}" failed. No retries permitted until 2026-07-15 17:28:08.28037413 +0000 UTC m=+1904.297787203 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9a9ae8b3-2b6b-47ec-98e4-c13603084168-maas-api-tls") pod "maas-api-e2e-ait-e7083808-66789766fb-flrh8" (UID: "9a9ae8b3-2b6b-47ec-98e4-c13603084168") : secret "maas-api-e2e-ait-e7083808-serving-cert" not found
Jul 15 17:28:07.780944 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:07.780925    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a9ae8b3-2b6b-47ec-98e4-c13603084168-openshift-service-ca\") pod \"maas-api-e2e-ait-e7083808-66789766fb-flrh8\" (UID: \"9a9ae8b3-2b6b-47ec-98e4-c13603084168\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8"
Jul 15 17:28:07.809575 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:07.809541    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcrv2\" (UniqueName: \"kubernetes.io/projected/9a9ae8b3-2b6b-47ec-98e4-c13603084168-kube-api-access-jcrv2\") pod \"maas-api-e2e-ait-e7083808-66789766fb-flrh8\" (UID: \"9a9ae8b3-2b6b-47ec-98e4-c13603084168\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8"
Jul 15 17:28:07.832965 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:07.832933    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb" event={"ID":"a699b5cf-fb45-4d08-8daa-cebc96714c68","Type":"ContainerStarted","Data":"a7e25ee2a0686685c6b8fce655d7943cf4948cb9d9b617980a77ca8f9c7f8e04"}
Jul 15 17:28:07.873293 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:07.873246    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb" podStartSLOduration=1.873230022 podStartE2EDuration="1.873230022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:28:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:28:07.871615732 +0000 UTC m=+1903.889028847" watchObservedRunningTime="2026-07-15 17:28:07.873230022 +0000 UTC m=+1903.890643117"
Jul 15 17:28:08.283861 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:08.283834    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a9ae8b3-2b6b-47ec-98e4-c13603084168-maas-api-tls\") pod \"maas-api-e2e-ait-e7083808-66789766fb-flrh8\" (UID: \"9a9ae8b3-2b6b-47ec-98e4-c13603084168\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8"
Jul 15 17:28:08.286294 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:08.286268    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a9ae8b3-2b6b-47ec-98e4-c13603084168-maas-api-tls\") pod \"maas-api-e2e-ait-e7083808-66789766fb-flrh8\" (UID: \"9a9ae8b3-2b6b-47ec-98e4-c13603084168\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8"
Jul 15 17:28:08.543249 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:08.543151    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8"
Jul 15 17:28:08.660062 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:08.660033    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:08.665116 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:08.665094    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:08.676091 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:08.675933    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8"]
Jul 15 17:28:08.678301 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:28:08.678279    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a9ae8b3_2b6b_47ec_98e4_c13603084168.slice/crio-663cf2d2824910ee28dceb94e7c1497d5d8ba128955ffc05f9ad419e1eb24287 WatchSource:0}: Error finding container 663cf2d2824910ee28dceb94e7c1497d5d8ba128955ffc05f9ad419e1eb24287: Status 404 returned error can't find the container with id 663cf2d2824910ee28dceb94e7c1497d5d8ba128955ffc05f9ad419e1eb24287
Jul 15 17:28:08.840117 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:08.839993    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8" event={"ID":"9a9ae8b3-2b6b-47ec-98e4-c13603084168","Type":"ContainerStarted","Data":"663cf2d2824910ee28dceb94e7c1497d5d8ba128955ffc05f9ad419e1eb24287"}
Jul 15 17:28:08.840314 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:08.840298    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:08.841435 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:08.841412    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:09.846676 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:09.846634    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8" event={"ID":"9a9ae8b3-2b6b-47ec-98e4-c13603084168","Type":"ContainerStarted","Data":"7466fa146ea145e202cd4048e6ecbf10d9fcf157c13f68c03dfda8dc398c9ad2"}
Jul 15 17:28:09.847136 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:09.846818    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8"
Jul 15 17:28:09.864405 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:09.864361    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8" podStartSLOduration=2.447554229 podStartE2EDuration="2.864345487s" totalImagesPullingTime="416.791258ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:28:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:28:09.86351294 +0000 UTC m=+1905.880926036" watchObservedRunningTime="2026-07-15 17:28:09.864345487 +0000 UTC m=+1905.881758582"
Jul 15 17:28:15.856601 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:15.856522    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8"
Jul 15 17:28:23.842196 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:23.842165    2575 patch_prober.go:28] interesting pod/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 15 17:28:23.842632 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:23.842230    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb" podUID="a699b5cf-fb45-4d08-8daa-cebc96714c68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 15 17:28:23.916782 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:23.916752    2575 generic.go:356] "Generic (PLEG): container finished" podID="a699b5cf-fb45-4d08-8daa-cebc96714c68" containerID="a7e25ee2a0686685c6b8fce655d7943cf4948cb9d9b617980a77ca8f9c7f8e04" exitCode=0
Jul 15 17:28:23.916943 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:23.916828    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb" event={"ID":"a699b5cf-fb45-4d08-8daa-cebc96714c68","Type":"ContainerDied","Data":"a7e25ee2a0686685c6b8fce655d7943cf4948cb9d9b617980a77ca8f9c7f8e04"}
Jul 15 17:28:23.917221 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:23.917208    2575 scope.go:117] "RemoveContainer" containerID="a7e25ee2a0686685c6b8fce655d7943cf4948cb9d9b617980a77ca8f9c7f8e04"
Jul 15 17:28:23.918230 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:23.918205    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:28:23.918323 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:23.918269    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:28:23.918323 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:23.918307    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:28:24.923825 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:24.923788    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb" event={"ID":"a699b5cf-fb45-4d08-8daa-cebc96714c68","Type":"ContainerStarted","Data":"67b02b5fa9e2d139ffa74096e42d1c50160e51c4a735d7c221a46bd48061ec80"}
Jul 15 17:28:25.660156 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:25.660104    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:25.660156 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:25.660145    2575 patch_prober.go:28] interesting pod/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 15 17:28:25.660409 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:25.660213    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb" podUID="a699b5cf-fb45-4d08-8daa-cebc96714c68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 15 17:28:26.662007 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:26.661963    2575 patch_prober.go:28] interesting pod/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 15 17:28:26.662510 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:26.662048    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb" podUID="a699b5cf-fb45-4d08-8daa-cebc96714c68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 15 17:28:27.659971 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:27.659932    2575 patch_prober.go:28] interesting pod/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 15 17:28:27.660281 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:27.660252    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb" podUID="a699b5cf-fb45-4d08-8daa-cebc96714c68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 15 17:28:28.660591 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:28.660530    2575 patch_prober.go:28] interesting pod/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 15 17:28:28.661175 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:28.661141    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb" podUID="a699b5cf-fb45-4d08-8daa-cebc96714c68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 15 17:28:29.664059 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:29.664031    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:29.664452 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:29.664311    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:28:29.664906 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:28:29.664890    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:29:13.386731 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:13.386694    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8"]
Jul 15 17:29:13.387312 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:13.386917    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8" podUID="9a9ae8b3-2b6b-47ec-98e4-c13603084168" containerName="maas-api" containerID="cri-o://7466fa146ea145e202cd4048e6ecbf10d9fcf157c13f68c03dfda8dc398c9ad2" gracePeriod=30
Jul 15 17:29:13.637810 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:13.637749    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8"
Jul 15 17:29:13.678094 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:13.678065    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a9ae8b3-2b6b-47ec-98e4-c13603084168-maas-api-tls\") pod \"9a9ae8b3-2b6b-47ec-98e4-c13603084168\" (UID: \"9a9ae8b3-2b6b-47ec-98e4-c13603084168\") "
Jul 15 17:29:13.678250 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:13.678112    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a9ae8b3-2b6b-47ec-98e4-c13603084168-openshift-service-ca\") pod \"9a9ae8b3-2b6b-47ec-98e4-c13603084168\" (UID: \"9a9ae8b3-2b6b-47ec-98e4-c13603084168\") "
Jul 15 17:29:13.678250 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:13.678196    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcrv2\" (UniqueName: \"kubernetes.io/projected/9a9ae8b3-2b6b-47ec-98e4-c13603084168-kube-api-access-jcrv2\") pod \"9a9ae8b3-2b6b-47ec-98e4-c13603084168\" (UID: \"9a9ae8b3-2b6b-47ec-98e4-c13603084168\") "
Jul 15 17:29:13.678491 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:13.678464    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a9ae8b3-2b6b-47ec-98e4-c13603084168-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9a9ae8b3-2b6b-47ec-98e4-c13603084168" (UID: "9a9ae8b3-2b6b-47ec-98e4-c13603084168"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:29:13.680115 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:13.680095    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a9ae8b3-2b6b-47ec-98e4-c13603084168-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9a9ae8b3-2b6b-47ec-98e4-c13603084168" (UID: "9a9ae8b3-2b6b-47ec-98e4-c13603084168"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:29:13.680230 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:13.680214    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a9ae8b3-2b6b-47ec-98e4-c13603084168-kube-api-access-jcrv2" (OuterVolumeSpecName: "kube-api-access-jcrv2") pod "9a9ae8b3-2b6b-47ec-98e4-c13603084168" (UID: "9a9ae8b3-2b6b-47ec-98e4-c13603084168"). InnerVolumeSpecName "kube-api-access-jcrv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:29:13.778919 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:13.778888    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a9ae8b3-2b6b-47ec-98e4-c13603084168-maas-api-tls\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:29:13.778919 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:13.778915    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a9ae8b3-2b6b-47ec-98e4-c13603084168-openshift-service-ca\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:29:13.778919 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:13.778925    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcrv2\" (UniqueName: \"kubernetes.io/projected/9a9ae8b3-2b6b-47ec-98e4-c13603084168-kube-api-access-jcrv2\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:29:14.162167 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:14.162131    2575 generic.go:356] "Generic (PLEG): container finished" podID="9a9ae8b3-2b6b-47ec-98e4-c13603084168" containerID="7466fa146ea145e202cd4048e6ecbf10d9fcf157c13f68c03dfda8dc398c9ad2" exitCode=0
Jul 15 17:29:14.162335 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:14.162175    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8"
Jul 15 17:29:14.162335 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:14.162215    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8" event={"ID":"9a9ae8b3-2b6b-47ec-98e4-c13603084168","Type":"ContainerDied","Data":"7466fa146ea145e202cd4048e6ecbf10d9fcf157c13f68c03dfda8dc398c9ad2"}
Jul 15 17:29:14.162335 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:14.162267    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8" event={"ID":"9a9ae8b3-2b6b-47ec-98e4-c13603084168","Type":"ContainerDied","Data":"663cf2d2824910ee28dceb94e7c1497d5d8ba128955ffc05f9ad419e1eb24287"}
Jul 15 17:29:14.162335 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:14.162289    2575 scope.go:117] "RemoveContainer" containerID="7466fa146ea145e202cd4048e6ecbf10d9fcf157c13f68c03dfda8dc398c9ad2"
Jul 15 17:29:14.171052 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:14.171034    2575 scope.go:117] "RemoveContainer" containerID="7466fa146ea145e202cd4048e6ecbf10d9fcf157c13f68c03dfda8dc398c9ad2"
Jul 15 17:29:14.171297 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:29:14.171278    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7466fa146ea145e202cd4048e6ecbf10d9fcf157c13f68c03dfda8dc398c9ad2\": container with ID starting with 7466fa146ea145e202cd4048e6ecbf10d9fcf157c13f68c03dfda8dc398c9ad2 not found: ID does not exist" containerID="7466fa146ea145e202cd4048e6ecbf10d9fcf157c13f68c03dfda8dc398c9ad2"
Jul 15 17:29:14.171342 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:14.171306    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7466fa146ea145e202cd4048e6ecbf10d9fcf157c13f68c03dfda8dc398c9ad2"} err="failed to get container status \"7466fa146ea145e202cd4048e6ecbf10d9fcf157c13f68c03dfda8dc398c9ad2\": rpc error: code = NotFound desc = could not find container \"7466fa146ea145e202cd4048e6ecbf10d9fcf157c13f68c03dfda8dc398c9ad2\": container with ID starting with 7466fa146ea145e202cd4048e6ecbf10d9fcf157c13f68c03dfda8dc398c9ad2 not found: ID does not exist"
Jul 15 17:29:14.183473 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:14.183448    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8"]
Jul 15 17:29:14.188723 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:14.188702    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e7083808-66789766fb-flrh8"]
Jul 15 17:29:14.526262 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:14.526186    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a9ae8b3-2b6b-47ec-98e4-c13603084168" path="/var/lib/kubelet/pods/9a9ae8b3-2b6b-47ec-98e4-c13603084168/volumes"
Jul 15 17:29:28.684710 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:28.684678    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"]
Jul 15 17:29:28.685181 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:28.684892    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb" podUID="a699b5cf-fb45-4d08-8daa-cebc96714c68" containerName="istio-proxy" containerID="cri-o://67b02b5fa9e2d139ffa74096e42d1c50160e51c4a735d7c221a46bd48061ec80" gracePeriod=30
Jul 15 17:29:29.667220 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:29.667177    2575 patch_prober.go:28] interesting pod/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 17:29:29.667388 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:29.667248    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb" podUID="a699b5cf-fb45-4d08-8daa-cebc96714c68" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 17:29:30.315196 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:30.315167    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d"]
Jul 15 17:29:30.315610 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:30.315594    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a9ae8b3-2b6b-47ec-98e4-c13603084168" containerName="maas-api"
Jul 15 17:29:30.315670 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:30.315612    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a9ae8b3-2b6b-47ec-98e4-c13603084168" containerName="maas-api"
Jul 15 17:29:30.315708 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:30.315695    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a9ae8b3-2b6b-47ec-98e4-c13603084168" containerName="maas-api"
Jul 15 17:29:30.318842 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:30.318788    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d"
Jul 15 17:29:30.321609 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:30.321542    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-65f681ff-serving-cert\""
Jul 15 17:29:30.331378 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:30.331353    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d"]
Jul 15 17:29:30.427940 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:30.427905    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/563398c6-d263-4a83-a8ef-28878bd761bb-maas-api-tls\") pod \"maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d\" (UID: \"563398c6-d263-4a83-a8ef-28878bd761bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d"
Jul 15 17:29:30.428145 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:30.427999    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/563398c6-d263-4a83-a8ef-28878bd761bb-openshift-service-ca\") pod \"maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d\" (UID: \"563398c6-d263-4a83-a8ef-28878bd761bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d"
Jul 15 17:29:30.428145 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:30.428072    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jkbp\" (UniqueName: \"kubernetes.io/projected/563398c6-d263-4a83-a8ef-28878bd761bb-kube-api-access-9jkbp\") pod \"maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d\" (UID: \"563398c6-d263-4a83-a8ef-28878bd761bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d"
Jul 15 17:29:30.529464 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:30.529434    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jkbp\" (UniqueName: \"kubernetes.io/projected/563398c6-d263-4a83-a8ef-28878bd761bb-kube-api-access-9jkbp\") pod \"maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d\" (UID: \"563398c6-d263-4a83-a8ef-28878bd761bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d"
Jul 15 17:29:30.529650 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:30.529495    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/563398c6-d263-4a83-a8ef-28878bd761bb-maas-api-tls\") pod \"maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d\" (UID: \"563398c6-d263-4a83-a8ef-28878bd761bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d"
Jul 15 17:29:30.529650 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:30.529534    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/563398c6-d263-4a83-a8ef-28878bd761bb-openshift-service-ca\") pod \"maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d\" (UID: \"563398c6-d263-4a83-a8ef-28878bd761bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d"
Jul 15 17:29:30.529650 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:29:30.529639    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-serving-cert: secret "maas-api-e2e-derive-65f681ff-serving-cert" not found
Jul 15 17:29:30.529798 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:29:30.529711    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/563398c6-d263-4a83-a8ef-28878bd761bb-maas-api-tls podName:563398c6-d263-4a83-a8ef-28878bd761bb nodeName:}" failed. No retries permitted until 2026-07-15 17:29:31.029689935 +0000 UTC m=+1987.047103020 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/563398c6-d263-4a83-a8ef-28878bd761bb-maas-api-tls") pod "maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d" (UID: "563398c6-d263-4a83-a8ef-28878bd761bb") : secret "maas-api-e2e-derive-65f681ff-serving-cert" not found
Jul 15 17:29:30.530107 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:30.530090    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/563398c6-d263-4a83-a8ef-28878bd761bb-openshift-service-ca\") pod \"maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d\" (UID: \"563398c6-d263-4a83-a8ef-28878bd761bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d"
Jul 15 17:29:30.543103 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:30.543075    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jkbp\" (UniqueName: \"kubernetes.io/projected/563398c6-d263-4a83-a8ef-28878bd761bb-kube-api-access-9jkbp\") pod \"maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d\" (UID: \"563398c6-d263-4a83-a8ef-28878bd761bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d"
Jul 15 17:29:31.034451 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:31.034412    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/563398c6-d263-4a83-a8ef-28878bd761bb-maas-api-tls\") pod \"maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d\" (UID: \"563398c6-d263-4a83-a8ef-28878bd761bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d"
Jul 15 17:29:31.036786 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:31.036763    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/563398c6-d263-4a83-a8ef-28878bd761bb-maas-api-tls\") pod \"maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d\" (UID: \"563398c6-d263-4a83-a8ef-28878bd761bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d"
Jul 15 17:29:31.229583 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:31.229529    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d"
Jul 15 17:29:31.373589 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:31.373540    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d"]
Jul 15 17:29:31.376200 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:29:31.376170    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod563398c6_d263_4a83_a8ef_28878bd761bb.slice/crio-79cccfe7cbee718a45bed515971d7d887cffe513a9a0f12612fb87d957928b81 WatchSource:0}: Error finding container 79cccfe7cbee718a45bed515971d7d887cffe513a9a0f12612fb87d957928b81: Status 404 returned error can't find the container with id 79cccfe7cbee718a45bed515971d7d887cffe513a9a0f12612fb87d957928b81
Jul 15 17:29:32.262990 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:32.262946    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d" event={"ID":"563398c6-d263-4a83-a8ef-28878bd761bb","Type":"ContainerStarted","Data":"61530dbece890efc7e5434311d8e8772a4763ab38cd32bfe5786df4a8fb47097"}
Jul 15 17:29:32.263195 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:32.263096    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d"
Jul 15 17:29:32.263195 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:32.263125    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d" event={"ID":"563398c6-d263-4a83-a8ef-28878bd761bb","Type":"ContainerStarted","Data":"79cccfe7cbee718a45bed515971d7d887cffe513a9a0f12612fb87d957928b81"}
Jul 15 17:29:32.285068 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:32.285017    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d" podStartSLOduration=1.883871456 podStartE2EDuration="2.285001991s" totalImagesPullingTime="401.130535ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:29:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:29:32.282334673 +0000 UTC m=+1988.299747794" watchObservedRunningTime="2026-07-15 17:29:32.285001991 +0000 UTC m=+1988.302415086"
Jul 15 17:29:33.946389 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:33.946325    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:29:34.062719 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.062688    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-token\") pod \"a699b5cf-fb45-4d08-8daa-cebc96714c68\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") "
Jul 15 17:29:34.062865 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.062772    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cktsb\" (UniqueName: \"kubernetes.io/projected/a699b5cf-fb45-4d08-8daa-cebc96714c68-kube-api-access-cktsb\") pod \"a699b5cf-fb45-4d08-8daa-cebc96714c68\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") "
Jul 15 17:29:34.062865 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.062824    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-credential-socket\") pod \"a699b5cf-fb45-4d08-8daa-cebc96714c68\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") "
Jul 15 17:29:34.062865 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.062840    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-workload-certs\") pod \"a699b5cf-fb45-4d08-8daa-cebc96714c68\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") "
Jul 15 17:29:34.062865 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.062858    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-envoy\") pod \"a699b5cf-fb45-4d08-8daa-cebc96714c68\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") "
Jul 15 17:29:34.063093 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.062880    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-podinfo\") pod \"a699b5cf-fb45-4d08-8daa-cebc96714c68\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") "
Jul 15 17:29:34.063093 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.062903    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-data\") pod \"a699b5cf-fb45-4d08-8daa-cebc96714c68\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") "
Jul 15 17:29:34.063093 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.062925    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a699b5cf-fb45-4d08-8daa-cebc96714c68-istiod-ca-cert\") pod \"a699b5cf-fb45-4d08-8daa-cebc96714c68\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") "
Jul 15 17:29:34.063093 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.062992    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-workload-socket\") pod \"a699b5cf-fb45-4d08-8daa-cebc96714c68\" (UID: \"a699b5cf-fb45-4d08-8daa-cebc96714c68\") "
Jul 15 17:29:34.063293 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.063215    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a699b5cf-fb45-4d08-8daa-cebc96714c68" (UID: "a699b5cf-fb45-4d08-8daa-cebc96714c68"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:29:34.063378 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.063343    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a699b5cf-fb45-4d08-8daa-cebc96714c68" (UID: "a699b5cf-fb45-4d08-8daa-cebc96714c68"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:29:34.063602 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.063545    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a699b5cf-fb45-4d08-8daa-cebc96714c68" (UID: "a699b5cf-fb45-4d08-8daa-cebc96714c68"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:29:34.063727 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.063598    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a699b5cf-fb45-4d08-8daa-cebc96714c68-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a699b5cf-fb45-4d08-8daa-cebc96714c68" (UID: "a699b5cf-fb45-4d08-8daa-cebc96714c68"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:29:34.063989 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.063962    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-data" (OuterVolumeSpecName: "istio-data") pod "a699b5cf-fb45-4d08-8daa-cebc96714c68" (UID: "a699b5cf-fb45-4d08-8daa-cebc96714c68"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:29:34.065463 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.065426    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a699b5cf-fb45-4d08-8daa-cebc96714c68" (UID: "a699b5cf-fb45-4d08-8daa-cebc96714c68"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:29:34.065463 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.065429    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a699b5cf-fb45-4d08-8daa-cebc96714c68" (UID: "a699b5cf-fb45-4d08-8daa-cebc96714c68"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 17:29:34.065633 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.065502    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a699b5cf-fb45-4d08-8daa-cebc96714c68-kube-api-access-cktsb" (OuterVolumeSpecName: "kube-api-access-cktsb") pod "a699b5cf-fb45-4d08-8daa-cebc96714c68" (UID: "a699b5cf-fb45-4d08-8daa-cebc96714c68"). InnerVolumeSpecName "kube-api-access-cktsb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:29:34.065633 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.065520    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-token" (OuterVolumeSpecName: "istio-token") pod "a699b5cf-fb45-4d08-8daa-cebc96714c68" (UID: "a699b5cf-fb45-4d08-8daa-cebc96714c68"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:29:34.164031 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.163988    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cktsb\" (UniqueName: \"kubernetes.io/projected/a699b5cf-fb45-4d08-8daa-cebc96714c68-kube-api-access-cktsb\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:29:34.164031 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.164023    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-credential-socket\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:29:34.164031 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.164037    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-workload-certs\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:29:34.164278 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.164050    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-envoy\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:29:34.164278 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.164058    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-podinfo\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:29:34.164278 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.164066    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-data\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:29:34.164278 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.164074    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a699b5cf-fb45-4d08-8daa-cebc96714c68-istiod-ca-cert\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:29:34.164278 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.164081    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a699b5cf-fb45-4d08-8daa-cebc96714c68-workload-socket\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:29:34.164278 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.164089    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a699b5cf-fb45-4d08-8daa-cebc96714c68-istio-token\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:29:34.275058 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.274976    2575 generic.go:356] "Generic (PLEG): container finished" podID="a699b5cf-fb45-4d08-8daa-cebc96714c68" containerID="67b02b5fa9e2d139ffa74096e42d1c50160e51c4a735d7c221a46bd48061ec80" exitCode=0
Jul 15 17:29:34.275058 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.275012    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb" event={"ID":"a699b5cf-fb45-4d08-8daa-cebc96714c68","Type":"ContainerDied","Data":"67b02b5fa9e2d139ffa74096e42d1c50160e51c4a735d7c221a46bd48061ec80"}
Jul 15 17:29:34.275058 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.275032    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"
Jul 15 17:29:34.275313 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.275082    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb" event={"ID":"a699b5cf-fb45-4d08-8daa-cebc96714c68","Type":"ContainerDied","Data":"5c01e1a5fb62448856ae78ebca6d2188bd083fc6c93781c2a5b6d1414fe0de76"}
Jul 15 17:29:34.275313 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.275103    2575 scope.go:117] "RemoveContainer" containerID="67b02b5fa9e2d139ffa74096e42d1c50160e51c4a735d7c221a46bd48061ec80"
Jul 15 17:29:34.284348 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.284331    2575 scope.go:117] "RemoveContainer" containerID="a7e25ee2a0686685c6b8fce655d7943cf4948cb9d9b617980a77ca8f9c7f8e04"
Jul 15 17:29:34.292329 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.292312    2575 scope.go:117] "RemoveContainer" containerID="67b02b5fa9e2d139ffa74096e42d1c50160e51c4a735d7c221a46bd48061ec80"
Jul 15 17:29:34.292586 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:29:34.292551    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67b02b5fa9e2d139ffa74096e42d1c50160e51c4a735d7c221a46bd48061ec80\": container with ID starting with 67b02b5fa9e2d139ffa74096e42d1c50160e51c4a735d7c221a46bd48061ec80 not found: ID does not exist" containerID="67b02b5fa9e2d139ffa74096e42d1c50160e51c4a735d7c221a46bd48061ec80"
Jul 15 17:29:34.292700 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.292593    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67b02b5fa9e2d139ffa74096e42d1c50160e51c4a735d7c221a46bd48061ec80"} err="failed to get container status \"67b02b5fa9e2d139ffa74096e42d1c50160e51c4a735d7c221a46bd48061ec80\": rpc error: code = NotFound desc = could not find container \"67b02b5fa9e2d139ffa74096e42d1c50160e51c4a735d7c221a46bd48061ec80\": container with ID starting with 67b02b5fa9e2d139ffa74096e42d1c50160e51c4a735d7c221a46bd48061ec80 not found: ID does not exist"
Jul 15 17:29:34.292700 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.292609    2575 scope.go:117] "RemoveContainer" containerID="a7e25ee2a0686685c6b8fce655d7943cf4948cb9d9b617980a77ca8f9c7f8e04"
Jul 15 17:29:34.292876 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:29:34.292861    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7e25ee2a0686685c6b8fce655d7943cf4948cb9d9b617980a77ca8f9c7f8e04\": container with ID starting with a7e25ee2a0686685c6b8fce655d7943cf4948cb9d9b617980a77ca8f9c7f8e04 not found: ID does not exist" containerID="a7e25ee2a0686685c6b8fce655d7943cf4948cb9d9b617980a77ca8f9c7f8e04"
Jul 15 17:29:34.292915 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.292879    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7e25ee2a0686685c6b8fce655d7943cf4948cb9d9b617980a77ca8f9c7f8e04"} err="failed to get container status \"a7e25ee2a0686685c6b8fce655d7943cf4948cb9d9b617980a77ca8f9c7f8e04\": rpc error: code = NotFound desc = could not find container \"a7e25ee2a0686685c6b8fce655d7943cf4948cb9d9b617980a77ca8f9c7f8e04\": container with ID starting with a7e25ee2a0686685c6b8fce655d7943cf4948cb9d9b617980a77ca8f9c7f8e04 not found: ID does not exist"
Jul 15 17:29:34.296854 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.296833    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"]
Jul 15 17:29:34.300249 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.300229    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e7083808-openshift-default-fc8b67769-74mgb"]
Jul 15 17:29:34.526729 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:34.526630    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a699b5cf-fb45-4d08-8daa-cebc96714c68" path="/var/lib/kubelet/pods/a699b5cf-fb45-4d08-8daa-cebc96714c68/volumes"
Jul 15 17:29:38.273201 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:29:38.273175    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d"
Jul 15 17:30:45.178832 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:30:45.178690    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d"]
Jul 15 17:30:45.179332 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:30:45.178944    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d" podUID="563398c6-d263-4a83-a8ef-28878bd761bb" containerName="maas-api" containerID="cri-o://61530dbece890efc7e5434311d8e8772a4763ab38cd32bfe5786df4a8fb47097" gracePeriod=30
Jul 15 17:30:45.434386 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:30:45.434326    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d"
Jul 15 17:30:45.484833 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:30:45.484802    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/563398c6-d263-4a83-a8ef-28878bd761bb-openshift-service-ca\") pod \"563398c6-d263-4a83-a8ef-28878bd761bb\" (UID: \"563398c6-d263-4a83-a8ef-28878bd761bb\") "
Jul 15 17:30:45.484993 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:30:45.484841    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/563398c6-d263-4a83-a8ef-28878bd761bb-maas-api-tls\") pod \"563398c6-d263-4a83-a8ef-28878bd761bb\" (UID: \"563398c6-d263-4a83-a8ef-28878bd761bb\") "
Jul 15 17:30:45.484993 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:30:45.484858    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jkbp\" (UniqueName: \"kubernetes.io/projected/563398c6-d263-4a83-a8ef-28878bd761bb-kube-api-access-9jkbp\") pod \"563398c6-d263-4a83-a8ef-28878bd761bb\" (UID: \"563398c6-d263-4a83-a8ef-28878bd761bb\") "
Jul 15 17:30:45.485209 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:30:45.485182    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/563398c6-d263-4a83-a8ef-28878bd761bb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "563398c6-d263-4a83-a8ef-28878bd761bb" (UID: "563398c6-d263-4a83-a8ef-28878bd761bb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:30:45.486962 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:30:45.486930    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/563398c6-d263-4a83-a8ef-28878bd761bb-kube-api-access-9jkbp" (OuterVolumeSpecName: "kube-api-access-9jkbp") pod "563398c6-d263-4a83-a8ef-28878bd761bb" (UID: "563398c6-d263-4a83-a8ef-28878bd761bb"). InnerVolumeSpecName "kube-api-access-9jkbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:30:45.487062 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:30:45.486966    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/563398c6-d263-4a83-a8ef-28878bd761bb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "563398c6-d263-4a83-a8ef-28878bd761bb" (UID: "563398c6-d263-4a83-a8ef-28878bd761bb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:30:45.585471 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:30:45.585436    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/563398c6-d263-4a83-a8ef-28878bd761bb-openshift-service-ca\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:30:45.585471 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:30:45.585467    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/563398c6-d263-4a83-a8ef-28878bd761bb-maas-api-tls\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:30:45.585471 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:30:45.585476    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jkbp\" (UniqueName: \"kubernetes.io/projected/563398c6-d263-4a83-a8ef-28878bd761bb-kube-api-access-9jkbp\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:30:45.618859 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:30:45.618828    2575 generic.go:356] "Generic (PLEG): container finished" podID="563398c6-d263-4a83-a8ef-28878bd761bb" containerID="61530dbece890efc7e5434311d8e8772a4763ab38cd32bfe5786df4a8fb47097" exitCode=0
Jul 15 17:30:45.619030 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:30:45.618873    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d"
Jul 15 17:30:45.619030 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:30:45.618918    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d" event={"ID":"563398c6-d263-4a83-a8ef-28878bd761bb","Type":"ContainerDied","Data":"61530dbece890efc7e5434311d8e8772a4763ab38cd32bfe5786df4a8fb47097"}
Jul 15 17:30:45.619030 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:30:45.618977    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d" event={"ID":"563398c6-d263-4a83-a8ef-28878bd761bb","Type":"ContainerDied","Data":"79cccfe7cbee718a45bed515971d7d887cffe513a9a0f12612fb87d957928b81"}
Jul 15 17:30:45.619030 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:30:45.619006    2575 scope.go:117] "RemoveContainer" containerID="61530dbece890efc7e5434311d8e8772a4763ab38cd32bfe5786df4a8fb47097"
Jul 15 17:30:45.628445 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:30:45.628423    2575 scope.go:117] "RemoveContainer" containerID="61530dbece890efc7e5434311d8e8772a4763ab38cd32bfe5786df4a8fb47097"
Jul 15 17:30:45.628728 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:30:45.628707    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61530dbece890efc7e5434311d8e8772a4763ab38cd32bfe5786df4a8fb47097\": container with ID starting with 61530dbece890efc7e5434311d8e8772a4763ab38cd32bfe5786df4a8fb47097 not found: ID does not exist" containerID="61530dbece890efc7e5434311d8e8772a4763ab38cd32bfe5786df4a8fb47097"
Jul 15 17:30:45.628786 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:30:45.628736    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61530dbece890efc7e5434311d8e8772a4763ab38cd32bfe5786df4a8fb47097"} err="failed to get container status \"61530dbece890efc7e5434311d8e8772a4763ab38cd32bfe5786df4a8fb47097\": rpc error: code = NotFound desc = could not find container \"61530dbece890efc7e5434311d8e8772a4763ab38cd32bfe5786df4a8fb47097\": container with ID starting with 61530dbece890efc7e5434311d8e8772a4763ab38cd32bfe5786df4a8fb47097 not found: ID does not exist"
Jul 15 17:30:45.640733 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:30:45.640709    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d"]
Jul 15 17:30:45.644731 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:30:45.644710    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-65f681ff-8f5b5dd4b-x884d"]
Jul 15 17:30:46.525430 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:30:46.525356    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="563398c6-d263-4a83-a8ef-28878bd761bb" path="/var/lib/kubelet/pods/563398c6-d263-4a83-a8ef-28878bd761bb/volumes"
Jul 15 17:31:24.540429 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:31:24.540403    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 17:31:24.545959 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:31:24.545939    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 17:32:41.644656 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.644615    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"]
Jul 15 17:32:41.645280 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.644981    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a699b5cf-fb45-4d08-8daa-cebc96714c68" containerName="istio-proxy"
Jul 15 17:32:41.645280 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.644992    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a699b5cf-fb45-4d08-8daa-cebc96714c68" containerName="istio-proxy"
Jul 15 17:32:41.645280 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.644999    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="563398c6-d263-4a83-a8ef-28878bd761bb" containerName="maas-api"
Jul 15 17:32:41.645280 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.645006    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="563398c6-d263-4a83-a8ef-28878bd761bb" containerName="maas-api"
Jul 15 17:32:41.645280 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.645026    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a699b5cf-fb45-4d08-8daa-cebc96714c68" containerName="istio-proxy"
Jul 15 17:32:41.645280 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.645032    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a699b5cf-fb45-4d08-8daa-cebc96714c68" containerName="istio-proxy"
Jul 15 17:32:41.645280 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.645082    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a699b5cf-fb45-4d08-8daa-cebc96714c68" containerName="istio-proxy"
Jul 15 17:32:41.645280 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.645090    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="563398c6-d263-4a83-a8ef-28878bd761bb" containerName="maas-api"
Jul 15 17:32:41.645280 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.645097    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a699b5cf-fb45-4d08-8daa-cebc96714c68" containerName="istio-proxy"
Jul 15 17:32:41.647889 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.647867    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.650352 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.650335    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-92183fb9-openshift-default-dockercfg-w725p\""
Jul 15 17:32:41.686677 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.686656    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"]
Jul 15 17:32:41.782377 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.782347    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cafad887-613c-4806-84be-866118a0d21d-istio-token\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.782609 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.782405    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cafad887-613c-4806-84be-866118a0d21d-istio-podinfo\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.782609 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.782464    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-workload-socket\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.782609 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.782492    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-workload-certs\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.782609 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.782526    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-istio-data\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.782609 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.782548    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r587b\" (UniqueName: \"kubernetes.io/projected/cafad887-613c-4806-84be-866118a0d21d-kube-api-access-r587b\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.782888 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.782617    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-istio-envoy\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.782888 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.782656    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-credential-socket\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.782888 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.782691    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cafad887-613c-4806-84be-866118a0d21d-istiod-ca-cert\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.883790 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.883753    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cafad887-613c-4806-84be-866118a0d21d-istio-podinfo\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.883964 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.883798    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-workload-socket\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.883964 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.883831    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-workload-certs\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.883964 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.883865    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-istio-data\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.883964 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.883887    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r587b\" (UniqueName: \"kubernetes.io/projected/cafad887-613c-4806-84be-866118a0d21d-kube-api-access-r587b\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.883964 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.883934    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-istio-envoy\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.884232 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.883973    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-credential-socket\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.884232 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.884008    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cafad887-613c-4806-84be-866118a0d21d-istiod-ca-cert\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.884232 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.884077    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cafad887-613c-4806-84be-866118a0d21d-istio-token\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.884377 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.884248    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-workload-certs\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.884377 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.884332    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-istio-data\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.884478 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.884374    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-credential-socket\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.884715 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.884694    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cafad887-613c-4806-84be-866118a0d21d-istiod-ca-cert\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.884813 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.884766    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-workload-socket\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.886675 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.886653    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-istio-envoy\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.886861 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.886842    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cafad887-613c-4806-84be-866118a0d21d-istio-podinfo\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.893380 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.893363    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cafad887-613c-4806-84be-866118a0d21d-istio-token\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.893812 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.893789    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r587b\" (UniqueName: \"kubernetes.io/projected/cafad887-613c-4806-84be-866118a0d21d-kube-api-access-r587b\") pod \"e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") " pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:41.958306 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:41.958248    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:42.101525 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:42.101501    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"]
Jul 15 17:32:42.103758 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:32:42.103731    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcafad887_613c_4806_84be_866118a0d21d.slice/crio-3fe211961832843ee99a1a973e41080522655102dbd51a68ef2e109c872c9138 WatchSource:0}: Error finding container 3fe211961832843ee99a1a973e41080522655102dbd51a68ef2e109c872c9138: Status 404 returned error can't find the container with id 3fe211961832843ee99a1a973e41080522655102dbd51a68ef2e109c872c9138
Jul 15 17:32:42.105982 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:42.105952    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:32:42.106088 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:42.106027    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:32:42.106088 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:42.106070    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:32:42.187198 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:42.187161    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl" event={"ID":"cafad887-613c-4806-84be-866118a0d21d","Type":"ContainerStarted","Data":"87c675f2f2ff39c79e9edddcca6c527a1ffbd8a84ec9e275895fd8be83ead474"}
Jul 15 17:32:42.187373 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:42.187216    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl" event={"ID":"cafad887-613c-4806-84be-866118a0d21d","Type":"ContainerStarted","Data":"3fe211961832843ee99a1a973e41080522655102dbd51a68ef2e109c872c9138"}
Jul 15 17:32:42.218545 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:42.218429    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl" podStartSLOduration=1.218405367 podStartE2EDuration="1.218405367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:32:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:32:42.21513783 +0000 UTC m=+2178.232550927" watchObservedRunningTime="2026-07-15 17:32:42.218405367 +0000 UTC m=+2178.235818463"
Jul 15 17:32:42.959161 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:42.959122    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:43.963343 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:43.963313    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:44.197240 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:44.197209    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:44.198040 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:44.198023    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:44.679193 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:44.679163    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d"]
Jul 15 17:32:44.682655 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:44.682633    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d"
Jul 15 17:32:44.686900 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:44.686880    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-92183fb9-serving-cert\""
Jul 15 17:32:44.705333 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:44.705306    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d"]
Jul 15 17:32:44.811577 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:44.811515    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsskn\" (UniqueName: \"kubernetes.io/projected/c7836c1f-2749-4872-a7d5-f5f61feddd9b-kube-api-access-qsskn\") pod \"maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d\" (UID: \"c7836c1f-2749-4872-a7d5-f5f61feddd9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d"
Jul 15 17:32:44.811759 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:44.811679    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7836c1f-2749-4872-a7d5-f5f61feddd9b-openshift-service-ca\") pod \"maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d\" (UID: \"c7836c1f-2749-4872-a7d5-f5f61feddd9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d"
Jul 15 17:32:44.811759 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:44.811718    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7836c1f-2749-4872-a7d5-f5f61feddd9b-maas-api-tls\") pod \"maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d\" (UID: \"c7836c1f-2749-4872-a7d5-f5f61feddd9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d"
Jul 15 17:32:44.912509 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:44.912480    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsskn\" (UniqueName: \"kubernetes.io/projected/c7836c1f-2749-4872-a7d5-f5f61feddd9b-kube-api-access-qsskn\") pod \"maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d\" (UID: \"c7836c1f-2749-4872-a7d5-f5f61feddd9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d"
Jul 15 17:32:44.912715 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:44.912586    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7836c1f-2749-4872-a7d5-f5f61feddd9b-openshift-service-ca\") pod \"maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d\" (UID: \"c7836c1f-2749-4872-a7d5-f5f61feddd9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d"
Jul 15 17:32:44.912715 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:44.912617    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7836c1f-2749-4872-a7d5-f5f61feddd9b-maas-api-tls\") pod \"maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d\" (UID: \"c7836c1f-2749-4872-a7d5-f5f61feddd9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d"
Jul 15 17:32:44.912833 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:32:44.912735    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-serving-cert: secret "maas-api-e2e-mt-92183fb9-serving-cert" not found
Jul 15 17:32:44.912833 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:32:44.912814    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7836c1f-2749-4872-a7d5-f5f61feddd9b-maas-api-tls podName:c7836c1f-2749-4872-a7d5-f5f61feddd9b nodeName:}" failed. No retries permitted until 2026-07-15 17:32:45.412791745 +0000 UTC m=+2181.430204819 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c7836c1f-2749-4872-a7d5-f5f61feddd9b-maas-api-tls") pod "maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d" (UID: "c7836c1f-2749-4872-a7d5-f5f61feddd9b") : secret "maas-api-e2e-mt-92183fb9-serving-cert" not found
Jul 15 17:32:44.913269 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:44.913247    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7836c1f-2749-4872-a7d5-f5f61feddd9b-openshift-service-ca\") pod \"maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d\" (UID: \"c7836c1f-2749-4872-a7d5-f5f61feddd9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d"
Jul 15 17:32:44.933288 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:44.933198    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsskn\" (UniqueName: \"kubernetes.io/projected/c7836c1f-2749-4872-a7d5-f5f61feddd9b-kube-api-access-qsskn\") pod \"maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d\" (UID: \"c7836c1f-2749-4872-a7d5-f5f61feddd9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d"
Jul 15 17:32:45.417801 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:45.417767    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7836c1f-2749-4872-a7d5-f5f61feddd9b-maas-api-tls\") pod \"maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d\" (UID: \"c7836c1f-2749-4872-a7d5-f5f61feddd9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d"
Jul 15 17:32:45.420196 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:45.420173    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7836c1f-2749-4872-a7d5-f5f61feddd9b-maas-api-tls\") pod \"maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d\" (UID: \"c7836c1f-2749-4872-a7d5-f5f61feddd9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d"
Jul 15 17:32:45.593705 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:45.593674    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d"
Jul 15 17:32:45.741901 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:45.741878    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d"]
Jul 15 17:32:45.744393 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:32:45.744360    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7836c1f_2749_4872_a7d5_f5f61feddd9b.slice/crio-457a11d6ff0c7c85bf7da9f45d658680c157cb8963183c6774ac63ec3f42b471 WatchSource:0}: Error finding container 457a11d6ff0c7c85bf7da9f45d658680c157cb8963183c6774ac63ec3f42b471: Status 404 returned error can't find the container with id 457a11d6ff0c7c85bf7da9f45d658680c157cb8963183c6774ac63ec3f42b471
Jul 15 17:32:45.746218 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:45.746195    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 17:32:46.208057 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:46.208025    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d" event={"ID":"c7836c1f-2749-4872-a7d5-f5f61feddd9b","Type":"ContainerStarted","Data":"457a11d6ff0c7c85bf7da9f45d658680c157cb8963183c6774ac63ec3f42b471"}
Jul 15 17:32:47.214839 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:47.214799    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d" event={"ID":"c7836c1f-2749-4872-a7d5-f5f61feddd9b","Type":"ContainerStarted","Data":"19f819945b811a4f7e01486ac76ab116ffe39fd121748df4ef85ea9f488a51da"}
Jul 15 17:32:47.215219 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:47.214879    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d"
Jul 15 17:32:47.241239 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:47.240922    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d" podStartSLOduration=2.505452979 podStartE2EDuration="3.240901937s" totalImagesPullingTime="735.448958ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:32:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:32:47.236797439 +0000 UTC m=+2183.254210536" watchObservedRunningTime="2026-07-15 17:32:47.240901937 +0000 UTC m=+2183.258315036"
Jul 15 17:32:53.225959 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:53.225932    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d"
Jul 15 17:32:57.272449 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:57.272339    2575 generic.go:356] "Generic (PLEG): container finished" podID="cafad887-613c-4806-84be-866118a0d21d" containerID="87c675f2f2ff39c79e9edddcca6c527a1ffbd8a84ec9e275895fd8be83ead474" exitCode=0
Jul 15 17:32:57.272449 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:57.272420    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl" event={"ID":"cafad887-613c-4806-84be-866118a0d21d","Type":"ContainerDied","Data":"87c675f2f2ff39c79e9edddcca6c527a1ffbd8a84ec9e275895fd8be83ead474"}
Jul 15 17:32:57.273042 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:57.272935    2575 scope.go:117] "RemoveContainer" containerID="87c675f2f2ff39c79e9edddcca6c527a1ffbd8a84ec9e275895fd8be83ead474"
Jul 15 17:32:57.274005 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:57.273978    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:32:57.274111 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:57.274056    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:32:57.274111 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:57.274103    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:32:58.279665 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:58.279628    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl" event={"ID":"cafad887-613c-4806-84be-866118a0d21d","Type":"ContainerStarted","Data":"cc0f5617f42ae663b8471ac77cca57867846fe7a1108d3dae11adeb9f157edcb"}
Jul 15 17:32:58.960755 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:58.960723    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:58.961464 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:58.961134    2575 patch_prober.go:28] interesting pod/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 15 17:32:58.961464 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:58.961203    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl" podUID="cafad887-613c-4806-84be-866118a0d21d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 15 17:32:59.198099 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:59.198069    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:32:59.960948 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:59.960906    2575 patch_prober.go:28] interesting pod/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 15 17:32:59.961425 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:32:59.960967    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl" podUID="cafad887-613c-4806-84be-866118a0d21d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 15 17:33:00.959330 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:00.959281    2575 patch_prober.go:28] interesting pod/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 15 17:33:00.959575 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:00.959354    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl" podUID="cafad887-613c-4806-84be-866118a0d21d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 15 17:33:01.958743 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:01.958697    2575 patch_prober.go:28] interesting pod/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 15 17:33:01.959215 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:01.958780    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl" podUID="cafad887-613c-4806-84be-866118a0d21d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 15 17:33:02.962770 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:02.962740    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:33:02.963944 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:02.963923    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:33:07.317591 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:07.317542    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d"]
Jul 15 17:33:07.318085 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:07.317814    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d" podUID="c7836c1f-2749-4872-a7d5-f5f61feddd9b" containerName="maas-api" containerID="cri-o://19f819945b811a4f7e01486ac76ab116ffe39fd121748df4ef85ea9f488a51da" gracePeriod=30
Jul 15 17:33:07.578334 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:07.578276    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d"
Jul 15 17:33:07.628957 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:07.628921    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qsskn\" (UniqueName: \"kubernetes.io/projected/c7836c1f-2749-4872-a7d5-f5f61feddd9b-kube-api-access-qsskn\") pod \"c7836c1f-2749-4872-a7d5-f5f61feddd9b\" (UID: \"c7836c1f-2749-4872-a7d5-f5f61feddd9b\") "
Jul 15 17:33:07.629104 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:07.629028    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7836c1f-2749-4872-a7d5-f5f61feddd9b-openshift-service-ca\") pod \"c7836c1f-2749-4872-a7d5-f5f61feddd9b\" (UID: \"c7836c1f-2749-4872-a7d5-f5f61feddd9b\") "
Jul 15 17:33:07.629104 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:07.629050    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7836c1f-2749-4872-a7d5-f5f61feddd9b-maas-api-tls\") pod \"c7836c1f-2749-4872-a7d5-f5f61feddd9b\" (UID: \"c7836c1f-2749-4872-a7d5-f5f61feddd9b\") "
Jul 15 17:33:07.629407 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:07.629381    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7836c1f-2749-4872-a7d5-f5f61feddd9b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c7836c1f-2749-4872-a7d5-f5f61feddd9b" (UID: "c7836c1f-2749-4872-a7d5-f5f61feddd9b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:33:07.630941 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:07.630915    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7836c1f-2749-4872-a7d5-f5f61feddd9b-kube-api-access-qsskn" (OuterVolumeSpecName: "kube-api-access-qsskn") pod "c7836c1f-2749-4872-a7d5-f5f61feddd9b" (UID: "c7836c1f-2749-4872-a7d5-f5f61feddd9b"). InnerVolumeSpecName "kube-api-access-qsskn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:33:07.631016 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:07.630990    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7836c1f-2749-4872-a7d5-f5f61feddd9b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c7836c1f-2749-4872-a7d5-f5f61feddd9b" (UID: "c7836c1f-2749-4872-a7d5-f5f61feddd9b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:33:07.729754 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:07.729725    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsskn\" (UniqueName: \"kubernetes.io/projected/c7836c1f-2749-4872-a7d5-f5f61feddd9b-kube-api-access-qsskn\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:33:07.729754 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:07.729750    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7836c1f-2749-4872-a7d5-f5f61feddd9b-openshift-service-ca\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:33:07.729754 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:07.729759    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7836c1f-2749-4872-a7d5-f5f61feddd9b-maas-api-tls\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:33:08.332111 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:08.332079    2575 generic.go:356] "Generic (PLEG): container finished" podID="c7836c1f-2749-4872-a7d5-f5f61feddd9b" containerID="19f819945b811a4f7e01486ac76ab116ffe39fd121748df4ef85ea9f488a51da" exitCode=0
Jul 15 17:33:08.332473 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:08.332122    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d"
Jul 15 17:33:08.332473 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:08.332121    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d" event={"ID":"c7836c1f-2749-4872-a7d5-f5f61feddd9b","Type":"ContainerDied","Data":"19f819945b811a4f7e01486ac76ab116ffe39fd121748df4ef85ea9f488a51da"}
Jul 15 17:33:08.332473 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:08.332247    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d" event={"ID":"c7836c1f-2749-4872-a7d5-f5f61feddd9b","Type":"ContainerDied","Data":"457a11d6ff0c7c85bf7da9f45d658680c157cb8963183c6774ac63ec3f42b471"}
Jul 15 17:33:08.332473 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:08.332275    2575 scope.go:117] "RemoveContainer" containerID="19f819945b811a4f7e01486ac76ab116ffe39fd121748df4ef85ea9f488a51da"
Jul 15 17:33:08.340963 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:08.340948    2575 scope.go:117] "RemoveContainer" containerID="19f819945b811a4f7e01486ac76ab116ffe39fd121748df4ef85ea9f488a51da"
Jul 15 17:33:08.341203 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:33:08.341188    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19f819945b811a4f7e01486ac76ab116ffe39fd121748df4ef85ea9f488a51da\": container with ID starting with 19f819945b811a4f7e01486ac76ab116ffe39fd121748df4ef85ea9f488a51da not found: ID does not exist" containerID="19f819945b811a4f7e01486ac76ab116ffe39fd121748df4ef85ea9f488a51da"
Jul 15 17:33:08.341249 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:08.341210    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19f819945b811a4f7e01486ac76ab116ffe39fd121748df4ef85ea9f488a51da"} err="failed to get container status \"19f819945b811a4f7e01486ac76ab116ffe39fd121748df4ef85ea9f488a51da\": rpc error: code = NotFound desc = could not find container \"19f819945b811a4f7e01486ac76ab116ffe39fd121748df4ef85ea9f488a51da\": container with ID starting with 19f819945b811a4f7e01486ac76ab116ffe39fd121748df4ef85ea9f488a51da not found: ID does not exist"
Jul 15 17:33:08.352685 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:08.352665    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d"]
Jul 15 17:33:08.356473 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:08.356454    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-92183fb9-545d5c4ff6-k9d8d"]
Jul 15 17:33:08.525493 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:08.525465    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7836c1f-2749-4872-a7d5-f5f61feddd9b" path="/var/lib/kubelet/pods/c7836c1f-2749-4872-a7d5-f5f61feddd9b/volumes"
Jul 15 17:33:22.576115 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:22.576082    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"]
Jul 15 17:33:22.576725 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:22.576364    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl" podUID="cafad887-613c-4806-84be-866118a0d21d" containerName="istio-proxy" containerID="cri-o://cc0f5617f42ae663b8471ac77cca57867846fe7a1108d3dae11adeb9f157edcb" gracePeriod=30
Jul 15 17:33:27.826695 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:27.826669    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:33:27.905961 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:27.905933    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-credential-socket\") pod \"cafad887-613c-4806-84be-866118a0d21d\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") "
Jul 15 17:33:27.906140 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:27.905972    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-istio-envoy\") pod \"cafad887-613c-4806-84be-866118a0d21d\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") "
Jul 15 17:33:27.906140 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:27.906008    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cafad887-613c-4806-84be-866118a0d21d-istiod-ca-cert\") pod \"cafad887-613c-4806-84be-866118a0d21d\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") "
Jul 15 17:33:27.906140 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:27.906054    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cafad887-613c-4806-84be-866118a0d21d-istio-token\") pod \"cafad887-613c-4806-84be-866118a0d21d\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") "
Jul 15 17:33:27.906140 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:27.906104    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cafad887-613c-4806-84be-866118a0d21d-istio-podinfo\") pod \"cafad887-613c-4806-84be-866118a0d21d\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") "
Jul 15 17:33:27.906375 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:27.906162    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-workload-socket\") pod \"cafad887-613c-4806-84be-866118a0d21d\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") "
Jul 15 17:33:27.906375 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:27.906187    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-workload-certs\") pod \"cafad887-613c-4806-84be-866118a0d21d\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") "
Jul 15 17:33:27.906375 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:27.906229    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r587b\" (UniqueName: \"kubernetes.io/projected/cafad887-613c-4806-84be-866118a0d21d-kube-api-access-r587b\") pod \"cafad887-613c-4806-84be-866118a0d21d\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") "
Jul 15 17:33:27.906375 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:27.906229    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cafad887-613c-4806-84be-866118a0d21d" (UID: "cafad887-613c-4806-84be-866118a0d21d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:33:27.906375 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:27.906253    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-istio-data\") pod \"cafad887-613c-4806-84be-866118a0d21d\" (UID: \"cafad887-613c-4806-84be-866118a0d21d\") "
Jul 15 17:33:27.906671 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:27.906452    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cafad887-613c-4806-84be-866118a0d21d" (UID: "cafad887-613c-4806-84be-866118a0d21d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:33:27.906671 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:27.906481    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cafad887-613c-4806-84be-866118a0d21d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cafad887-613c-4806-84be-866118a0d21d" (UID: "cafad887-613c-4806-84be-866118a0d21d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:33:27.906782 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:27.906689    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cafad887-613c-4806-84be-866118a0d21d-istiod-ca-cert\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:33:27.906782 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:27.906710    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-workload-certs\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:33:27.906782 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:27.906724    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-credential-socket\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:33:27.906782 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:27.906719    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cafad887-613c-4806-84be-866118a0d21d" (UID: "cafad887-613c-4806-84be-866118a0d21d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:33:27.907080 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:27.907061    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-istio-data" (OuterVolumeSpecName: "istio-data") pod "cafad887-613c-4806-84be-866118a0d21d" (UID: "cafad887-613c-4806-84be-866118a0d21d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:33:27.908269 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:27.908245    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cafad887-613c-4806-84be-866118a0d21d" (UID: "cafad887-613c-4806-84be-866118a0d21d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:33:27.908376 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:27.908299    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cafad887-613c-4806-84be-866118a0d21d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cafad887-613c-4806-84be-866118a0d21d" (UID: "cafad887-613c-4806-84be-866118a0d21d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 17:33:27.908571 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:27.908534    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cafad887-613c-4806-84be-866118a0d21d-istio-token" (OuterVolumeSpecName: "istio-token") pod "cafad887-613c-4806-84be-866118a0d21d" (UID: "cafad887-613c-4806-84be-866118a0d21d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:33:27.908679 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:27.908649    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cafad887-613c-4806-84be-866118a0d21d-kube-api-access-r587b" (OuterVolumeSpecName: "kube-api-access-r587b") pod "cafad887-613c-4806-84be-866118a0d21d" (UID: "cafad887-613c-4806-84be-866118a0d21d"). InnerVolumeSpecName "kube-api-access-r587b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:33:28.007465 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:28.007432    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-workload-socket\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:33:28.007465 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:28.007461    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r587b\" (UniqueName: \"kubernetes.io/projected/cafad887-613c-4806-84be-866118a0d21d-kube-api-access-r587b\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:33:28.007687 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:28.007474    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-istio-data\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:33:28.007687 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:28.007486    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cafad887-613c-4806-84be-866118a0d21d-istio-envoy\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:33:28.007687 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:28.007497    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cafad887-613c-4806-84be-866118a0d21d-istio-token\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:33:28.007687 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:28.007508    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cafad887-613c-4806-84be-866118a0d21d-istio-podinfo\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:33:28.429440 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:28.429410    2575 generic.go:356] "Generic (PLEG): container finished" podID="cafad887-613c-4806-84be-866118a0d21d" containerID="cc0f5617f42ae663b8471ac77cca57867846fe7a1108d3dae11adeb9f157edcb" exitCode=0
Jul 15 17:33:28.429632 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:28.429458    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"
Jul 15 17:33:28.429632 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:28.429499    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl" event={"ID":"cafad887-613c-4806-84be-866118a0d21d","Type":"ContainerDied","Data":"cc0f5617f42ae663b8471ac77cca57867846fe7a1108d3dae11adeb9f157edcb"}
Jul 15 17:33:28.429632 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:28.429550    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl" event={"ID":"cafad887-613c-4806-84be-866118a0d21d","Type":"ContainerDied","Data":"3fe211961832843ee99a1a973e41080522655102dbd51a68ef2e109c872c9138"}
Jul 15 17:33:28.429632 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:28.429592    2575 scope.go:117] "RemoveContainer" containerID="cc0f5617f42ae663b8471ac77cca57867846fe7a1108d3dae11adeb9f157edcb"
Jul 15 17:33:28.438764 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:28.438748    2575 scope.go:117] "RemoveContainer" containerID="87c675f2f2ff39c79e9edddcca6c527a1ffbd8a84ec9e275895fd8be83ead474"
Jul 15 17:33:28.448940 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:28.448922    2575 scope.go:117] "RemoveContainer" containerID="cc0f5617f42ae663b8471ac77cca57867846fe7a1108d3dae11adeb9f157edcb"
Jul 15 17:33:28.449268 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:33:28.449239    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc0f5617f42ae663b8471ac77cca57867846fe7a1108d3dae11adeb9f157edcb\": container with ID starting with cc0f5617f42ae663b8471ac77cca57867846fe7a1108d3dae11adeb9f157edcb not found: ID does not exist" containerID="cc0f5617f42ae663b8471ac77cca57867846fe7a1108d3dae11adeb9f157edcb"
Jul 15 17:33:28.449357 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:28.449278    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc0f5617f42ae663b8471ac77cca57867846fe7a1108d3dae11adeb9f157edcb"} err="failed to get container status \"cc0f5617f42ae663b8471ac77cca57867846fe7a1108d3dae11adeb9f157edcb\": rpc error: code = NotFound desc = could not find container \"cc0f5617f42ae663b8471ac77cca57867846fe7a1108d3dae11adeb9f157edcb\": container with ID starting with cc0f5617f42ae663b8471ac77cca57867846fe7a1108d3dae11adeb9f157edcb not found: ID does not exist"
Jul 15 17:33:28.449357 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:28.449303    2575 scope.go:117] "RemoveContainer" containerID="87c675f2f2ff39c79e9edddcca6c527a1ffbd8a84ec9e275895fd8be83ead474"
Jul 15 17:33:28.449773 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:33:28.449749    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87c675f2f2ff39c79e9edddcca6c527a1ffbd8a84ec9e275895fd8be83ead474\": container with ID starting with 87c675f2f2ff39c79e9edddcca6c527a1ffbd8a84ec9e275895fd8be83ead474 not found: ID does not exist" containerID="87c675f2f2ff39c79e9edddcca6c527a1ffbd8a84ec9e275895fd8be83ead474"
Jul 15 17:33:28.449883 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:28.449779    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87c675f2f2ff39c79e9edddcca6c527a1ffbd8a84ec9e275895fd8be83ead474"} err="failed to get container status \"87c675f2f2ff39c79e9edddcca6c527a1ffbd8a84ec9e275895fd8be83ead474\": rpc error: code = NotFound desc = could not find container \"87c675f2f2ff39c79e9edddcca6c527a1ffbd8a84ec9e275895fd8be83ead474\": container with ID starting with 87c675f2f2ff39c79e9edddcca6c527a1ffbd8a84ec9e275895fd8be83ead474 not found: ID does not exist"
Jul 15 17:33:28.451917 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:28.451897    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"]
Jul 15 17:33:28.455257 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:28.455238    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-92183fb9-openshift-default-5b774bffdc-9d4jl"]
Jul 15 17:33:28.525658 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:28.525630    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cafad887-613c-4806-84be-866118a0d21d" path="/var/lib/kubelet/pods/cafad887-613c-4806-84be-866118a0d21d/volumes"
Jul 15 17:33:42.697139 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.697107    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"]
Jul 15 17:33:42.697506 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.697485    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cafad887-613c-4806-84be-866118a0d21d" containerName="istio-proxy"
Jul 15 17:33:42.697506 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.697497    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="cafad887-613c-4806-84be-866118a0d21d" containerName="istio-proxy"
Jul 15 17:33:42.697506 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.697507    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cafad887-613c-4806-84be-866118a0d21d" containerName="istio-proxy"
Jul 15 17:33:42.697630 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.697513    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="cafad887-613c-4806-84be-866118a0d21d" containerName="istio-proxy"
Jul 15 17:33:42.697630 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.697522    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7836c1f-2749-4872-a7d5-f5f61feddd9b" containerName="maas-api"
Jul 15 17:33:42.697630 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.697529    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7836c1f-2749-4872-a7d5-f5f61feddd9b" containerName="maas-api"
Jul 15 17:33:42.697630 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.697601    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7836c1f-2749-4872-a7d5-f5f61feddd9b" containerName="maas-api"
Jul 15 17:33:42.697630 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.697613    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="cafad887-613c-4806-84be-866118a0d21d" containerName="istio-proxy"
Jul 15 17:33:42.697630 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.697619    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="cafad887-613c-4806-84be-866118a0d21d" containerName="istio-proxy"
Jul 15 17:33:42.700636 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.700619    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.703615 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.703597    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-427b7c64-openshift-default-dockercfg-x46rz\""
Jul 15 17:33:42.729073 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.729051    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"]
Jul 15 17:33:42.836993 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.836955    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-credential-socket\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.837196 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.837001    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2k2h\" (UniqueName: \"kubernetes.io/projected/e0f951ae-739a-4281-9b58-c202914a5c67-kube-api-access-d2k2h\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.837196 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.837072    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0f951ae-739a-4281-9b58-c202914a5c67-istiod-ca-cert\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.837196 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.837107    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-workload-socket\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.837196 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.837142    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0f951ae-739a-4281-9b58-c202914a5c67-istio-token\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.837398 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.837206    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0f951ae-739a-4281-9b58-c202914a5c67-istio-podinfo\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.837398 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.837276    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-istio-envoy\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.837398 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.837313    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-workload-certs\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.837398 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.837344    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-istio-data\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.938410 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.938376    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-credential-socket\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.938410 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.938410    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2k2h\" (UniqueName: \"kubernetes.io/projected/e0f951ae-739a-4281-9b58-c202914a5c67-kube-api-access-d2k2h\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.938668 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.938441    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0f951ae-739a-4281-9b58-c202914a5c67-istiod-ca-cert\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.938668 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.938469    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-workload-socket\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.938668 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.938505    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0f951ae-739a-4281-9b58-c202914a5c67-istio-token\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.938668 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.938546    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0f951ae-739a-4281-9b58-c202914a5c67-istio-podinfo\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.938668 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.938593    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-istio-envoy\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.938668 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.938623    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-workload-certs\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.938668 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.938647    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-istio-data\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.939066 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.938895    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-credential-socket\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.939066 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.938932    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-workload-socket\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.939066 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.939004    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-istio-data\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.939186 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.939084    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-workload-certs\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.939253 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.939235    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0f951ae-739a-4281-9b58-c202914a5c67-istiod-ca-cert\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.941090 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.941063    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-istio-envoy\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.941336 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.941318    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0f951ae-739a-4281-9b58-c202914a5c67-istio-podinfo\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.950818 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.950758    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2k2h\" (UniqueName: \"kubernetes.io/projected/e0f951ae-739a-4281-9b58-c202914a5c67-kube-api-access-d2k2h\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:42.950818 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:42.950790    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0f951ae-739a-4281-9b58-c202914a5c67-istio-token\") pod \"e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") " pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:43.010565 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:43.010539    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:43.149821 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:43.149789    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"]
Jul 15 17:33:43.153045 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:33:43.153020    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0f951ae_739a_4281_9b58_c202914a5c67.slice/crio-ad505e4a4366b17070bdc31c14811f84456b0fe14c3e21e35cdd57a192443d05 WatchSource:0}: Error finding container ad505e4a4366b17070bdc31c14811f84456b0fe14c3e21e35cdd57a192443d05: Status 404 returned error can't find the container with id ad505e4a4366b17070bdc31c14811f84456b0fe14c3e21e35cdd57a192443d05
Jul 15 17:33:43.155222 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:43.155182    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:33:43.155300 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:43.155262    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:33:43.155300 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:43.155296    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:33:43.501508 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:43.501424    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk" event={"ID":"e0f951ae-739a-4281-9b58-c202914a5c67","Type":"ContainerStarted","Data":"cbb60e4971de5ecb1b16796958b5bd76d6a68ddfab9508a8fe582efe35e979e2"}
Jul 15 17:33:43.501508 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:43.501477    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk" event={"ID":"e0f951ae-739a-4281-9b58-c202914a5c67","Type":"ContainerStarted","Data":"ad505e4a4366b17070bdc31c14811f84456b0fe14c3e21e35cdd57a192443d05"}
Jul 15 17:33:43.537829 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:43.537784    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk" podStartSLOduration=1.5377675800000001 podStartE2EDuration="1.53776758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:33:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:33:43.536723237 +0000 UTC m=+2239.554136333" watchObservedRunningTime="2026-07-15 17:33:43.53776758 +0000 UTC m=+2239.555180676"
Jul 15 17:33:44.010808 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:44.010778    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:44.015623 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:44.015600    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:44.507052 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:44.507024    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:44.508098 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:44.508081    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:33:45.544444 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:45.544406    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld"]
Jul 15 17:33:45.547422 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:45.547403    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld"
Jul 15 17:33:45.549969 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:45.549941    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-427b7c64-serving-cert\""
Jul 15 17:33:45.558054 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:45.558031    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld"]
Jul 15 17:33:45.560436 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:45.560412    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b55db003-49f0-437d-a303-0f3d39fe9e68-openshift-service-ca\") pod \"maas-api-e2e-mt-427b7c64-685687ff65-qq2ld\" (UID: \"b55db003-49f0-437d-a303-0f3d39fe9e68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld"
Jul 15 17:33:45.560583 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:45.560549    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55db003-49f0-437d-a303-0f3d39fe9e68-maas-api-tls\") pod \"maas-api-e2e-mt-427b7c64-685687ff65-qq2ld\" (UID: \"b55db003-49f0-437d-a303-0f3d39fe9e68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld"
Jul 15 17:33:45.560658 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:45.560627    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fwch\" (UniqueName: \"kubernetes.io/projected/b55db003-49f0-437d-a303-0f3d39fe9e68-kube-api-access-2fwch\") pod \"maas-api-e2e-mt-427b7c64-685687ff65-qq2ld\" (UID: \"b55db003-49f0-437d-a303-0f3d39fe9e68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld"
Jul 15 17:33:45.661847 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:45.661816    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55db003-49f0-437d-a303-0f3d39fe9e68-maas-api-tls\") pod \"maas-api-e2e-mt-427b7c64-685687ff65-qq2ld\" (UID: \"b55db003-49f0-437d-a303-0f3d39fe9e68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld"
Jul 15 17:33:45.662040 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:45.661869    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fwch\" (UniqueName: \"kubernetes.io/projected/b55db003-49f0-437d-a303-0f3d39fe9e68-kube-api-access-2fwch\") pod \"maas-api-e2e-mt-427b7c64-685687ff65-qq2ld\" (UID: \"b55db003-49f0-437d-a303-0f3d39fe9e68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld"
Jul 15 17:33:45.662040 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:45.661921    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b55db003-49f0-437d-a303-0f3d39fe9e68-openshift-service-ca\") pod \"maas-api-e2e-mt-427b7c64-685687ff65-qq2ld\" (UID: \"b55db003-49f0-437d-a303-0f3d39fe9e68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld"
Jul 15 17:33:45.662040 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:33:45.661989    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-serving-cert: secret "maas-api-e2e-mt-427b7c64-serving-cert" not found
Jul 15 17:33:45.662218 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:33:45.662064    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b55db003-49f0-437d-a303-0f3d39fe9e68-maas-api-tls podName:b55db003-49f0-437d-a303-0f3d39fe9e68 nodeName:}" failed. No retries permitted until 2026-07-15 17:33:46.162044349 +0000 UTC m=+2242.179457427 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b55db003-49f0-437d-a303-0f3d39fe9e68-maas-api-tls") pod "maas-api-e2e-mt-427b7c64-685687ff65-qq2ld" (UID: "b55db003-49f0-437d-a303-0f3d39fe9e68") : secret "maas-api-e2e-mt-427b7c64-serving-cert" not found
Jul 15 17:33:45.662785 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:45.662760    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b55db003-49f0-437d-a303-0f3d39fe9e68-openshift-service-ca\") pod \"maas-api-e2e-mt-427b7c64-685687ff65-qq2ld\" (UID: \"b55db003-49f0-437d-a303-0f3d39fe9e68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld"
Jul 15 17:33:45.678097 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:45.678076    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fwch\" (UniqueName: \"kubernetes.io/projected/b55db003-49f0-437d-a303-0f3d39fe9e68-kube-api-access-2fwch\") pod \"maas-api-e2e-mt-427b7c64-685687ff65-qq2ld\" (UID: \"b55db003-49f0-437d-a303-0f3d39fe9e68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld"
Jul 15 17:33:46.169919 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:46.167326    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55db003-49f0-437d-a303-0f3d39fe9e68-maas-api-tls\") pod \"maas-api-e2e-mt-427b7c64-685687ff65-qq2ld\" (UID: \"b55db003-49f0-437d-a303-0f3d39fe9e68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld"
Jul 15 17:33:46.173030 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:46.172993    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55db003-49f0-437d-a303-0f3d39fe9e68-maas-api-tls\") pod \"maas-api-e2e-mt-427b7c64-685687ff65-qq2ld\" (UID: \"b55db003-49f0-437d-a303-0f3d39fe9e68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld"
Jul 15 17:33:46.459511 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:46.459431    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld"
Jul 15 17:33:46.810537 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:46.810510    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld"]
Jul 15 17:33:46.812179 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:33:46.812147    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb55db003_49f0_437d_a303_0f3d39fe9e68.slice/crio-8803ccfc396c379f97f4d29f43afc4b94f92ba9598759aa3c6aa2b883472d72f WatchSource:0}: Error finding container 8803ccfc396c379f97f4d29f43afc4b94f92ba9598759aa3c6aa2b883472d72f: Status 404 returned error can't find the container with id 8803ccfc396c379f97f4d29f43afc4b94f92ba9598759aa3c6aa2b883472d72f
Jul 15 17:33:47.524215 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:47.524163    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld" event={"ID":"b55db003-49f0-437d-a303-0f3d39fe9e68","Type":"ContainerStarted","Data":"ae353ce80e79af8981bfec5269e2c78ae1ba28ec16ca81bc0d7da77c2f2ebc0d"}
Jul 15 17:33:47.524399 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:47.524266    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld"
Jul 15 17:33:47.524399 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:47.524294    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld" event={"ID":"b55db003-49f0-437d-a303-0f3d39fe9e68","Type":"ContainerStarted","Data":"8803ccfc396c379f97f4d29f43afc4b94f92ba9598759aa3c6aa2b883472d72f"}
Jul 15 17:33:47.546711 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:47.546621    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld" podStartSLOduration=2.082283421 podStartE2EDuration="2.546606744s" totalImagesPullingTime="464.323323ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:33:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:33:47.545358433 +0000 UTC m=+2243.562771569" watchObservedRunningTime="2026-07-15 17:33:47.546606744 +0000 UTC m=+2243.564019840"
Jul 15 17:33:53.534064 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:53.534035    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld"
Jul 15 17:33:59.508819 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:59.508782    2575 patch_prober.go:28] interesting pod/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 15 17:33:59.509237 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:59.508845    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk" podUID="e0f951ae-739a-4281-9b58-c202914a5c67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 15 17:33:59.602256 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:59.602225    2575 generic.go:356] "Generic (PLEG): container finished" podID="e0f951ae-739a-4281-9b58-c202914a5c67" containerID="cbb60e4971de5ecb1b16796958b5bd76d6a68ddfab9508a8fe582efe35e979e2" exitCode=0
Jul 15 17:33:59.602407 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:59.602297    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk" event={"ID":"e0f951ae-739a-4281-9b58-c202914a5c67","Type":"ContainerDied","Data":"cbb60e4971de5ecb1b16796958b5bd76d6a68ddfab9508a8fe582efe35e979e2"}
Jul 15 17:33:59.602693 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:59.602679    2575 scope.go:117] "RemoveContainer" containerID="cbb60e4971de5ecb1b16796958b5bd76d6a68ddfab9508a8fe582efe35e979e2"
Jul 15 17:33:59.603761 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:59.603734    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:33:59.603839 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:59.603821    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:33:59.603878 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:33:59.603867    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:34:00.609374 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:00.609336    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk" event={"ID":"e0f951ae-739a-4281-9b58-c202914a5c67","Type":"ContainerStarted","Data":"cd00e7a1b24c1774d4d42710739bf4e02ddd8a9738d3ec39b06f5e42795d741c"}
Jul 15 17:34:01.011391 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:01.011353    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:34:01.011590 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:01.011537    2575 patch_prober.go:28] interesting pod/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 15 17:34:01.011646 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:01.011614    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk" podUID="e0f951ae-739a-4281-9b58-c202914a5c67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 15 17:34:02.011639 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:02.011606    2575 patch_prober.go:28] interesting pod/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 15 17:34:02.012037 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:02.011659    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk" podUID="e0f951ae-739a-4281-9b58-c202914a5c67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 15 17:34:03.011832 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:03.011798    2575 patch_prober.go:28] interesting pod/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 15 17:34:03.012202 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:03.011852    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk" podUID="e0f951ae-739a-4281-9b58-c202914a5c67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 15 17:34:04.011650 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:04.011614    2575 patch_prober.go:28] interesting pod/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 15 17:34:04.011842 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:04.011673    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk" podUID="e0f951ae-739a-4281-9b58-c202914a5c67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 15 17:34:05.014669 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:05.014634    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:34:05.015033 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:05.014891    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:34:05.015595 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:05.015578    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:34:10.411158 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.411125    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"]
Jul 15 17:34:10.414075 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.414052    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:10.417668 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.417642    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-427b7c64\"/\"default-dockercfg-dgf7c\""
Jul 15 17:34:10.417787 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.417683    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-427b7c64\"/\"openshift-service-ca.crt\""
Jul 15 17:34:10.417787 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.417748    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-427b7c64\"/\"e2e-lifecycle-model-427b7c64-kserve-self-signed-certs\""
Jul 15 17:34:10.417883 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.417754    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-427b7c64\"/\"kube-root-ca.crt\""
Jul 15 17:34:10.428722 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.427513    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"]
Jul 15 17:34:10.493495 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.493468    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-model-cache\") pod \"e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") " pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:10.493665 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.493514    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/772b7ac0-f909-415a-8642-5976ce3e6d5e-tls-certs\") pod \"e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") " pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:10.493712 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.493658    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-home\") pod \"e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") " pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:10.493712 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.493705    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-tmp-dir\") pod \"e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") " pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:10.493819 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.493764    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhjps\" (UniqueName: \"kubernetes.io/projected/772b7ac0-f909-415a-8642-5976ce3e6d5e-kube-api-access-bhjps\") pod \"e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") " pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:10.493878 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.493812    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-dshm\") pod \"e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") " pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:10.594983 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.594949    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/772b7ac0-f909-415a-8642-5976ce3e6d5e-tls-certs\") pod \"e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") " pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:10.595158 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.595049    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-home\") pod \"e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") " pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:10.595158 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.595086    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-tmp-dir\") pod \"e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") " pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:10.595158 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.595119    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhjps\" (UniqueName: \"kubernetes.io/projected/772b7ac0-f909-415a-8642-5976ce3e6d5e-kube-api-access-bhjps\") pod \"e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") " pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:10.595158 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.595149    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-dshm\") pod \"e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") " pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:10.595374 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.595182    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-model-cache\") pod \"e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") " pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:10.595623 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.595593    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-tmp-dir\") pod \"e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") " pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:10.595623 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.595612    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-home\") pod \"e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") " pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:10.595797 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.595643    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-model-cache\") pod \"e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") " pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:10.597413 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.597388    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-dshm\") pod \"e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") " pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:10.597413 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.597411    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/772b7ac0-f909-415a-8642-5976ce3e6d5e-tls-certs\") pod \"e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") " pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:10.602970 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.602950    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhjps\" (UniqueName: \"kubernetes.io/projected/772b7ac0-f909-415a-8642-5976ce3e6d5e-kube-api-access-bhjps\") pod \"e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") " pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:10.723499 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.723412    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:10.862606 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:10.862581    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"]
Jul 15 17:34:10.864774 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:34:10.864743    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod772b7ac0_f909_415a_8642_5976ce3e6d5e.slice/crio-aa6407f7cf43443f41945456f1248220eb2aa4aee4d136efc297eead5823870f WatchSource:0}: Error finding container aa6407f7cf43443f41945456f1248220eb2aa4aee4d136efc297eead5823870f: Status 404 returned error can't find the container with id aa6407f7cf43443f41945456f1248220eb2aa4aee4d136efc297eead5823870f
Jul 15 17:34:11.666971 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:11.666932    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl" event={"ID":"772b7ac0-f909-415a-8642-5976ce3e6d5e","Type":"ContainerStarted","Data":"8ee0e8dde8138a3b24b9248e1fcc453bb44ad927cadf81fac2cd19a4e67cbc08"}
Jul 15 17:34:11.667340 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:11.666983    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl" event={"ID":"772b7ac0-f909-415a-8642-5976ce3e6d5e","Type":"ContainerStarted","Data":"aa6407f7cf43443f41945456f1248220eb2aa4aee4d136efc297eead5823870f"}
Jul 15 17:34:11.686023 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:11.685977    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl" podStartSLOduration=1.685962896 podStartE2EDuration="1.685962896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:34:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:34:11.684405871 +0000 UTC m=+2267.701818969" watchObservedRunningTime="2026-07-15 17:34:11.685962896 +0000 UTC m=+2267.703375992"
Jul 15 17:34:20.724159 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:20.724116    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:20.724674 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:20.724172    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:20.736773 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:20.736751    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:21.724789 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:21.724765    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:34:48.934441 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:48.934404    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld"]
Jul 15 17:34:48.935001 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:48.934713    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld" podUID="b55db003-49f0-437d-a303-0f3d39fe9e68" containerName="maas-api" containerID="cri-o://ae353ce80e79af8981bfec5269e2c78ae1ba28ec16ca81bc0d7da77c2f2ebc0d" gracePeriod=30
Jul 15 17:34:49.186416 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:49.186362    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld"
Jul 15 17:34:49.244075 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:49.244047    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55db003-49f0-437d-a303-0f3d39fe9e68-maas-api-tls\") pod \"b55db003-49f0-437d-a303-0f3d39fe9e68\" (UID: \"b55db003-49f0-437d-a303-0f3d39fe9e68\") "
Jul 15 17:34:49.244226 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:49.244092    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fwch\" (UniqueName: \"kubernetes.io/projected/b55db003-49f0-437d-a303-0f3d39fe9e68-kube-api-access-2fwch\") pod \"b55db003-49f0-437d-a303-0f3d39fe9e68\" (UID: \"b55db003-49f0-437d-a303-0f3d39fe9e68\") "
Jul 15 17:34:49.244226 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:49.244188    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b55db003-49f0-437d-a303-0f3d39fe9e68-openshift-service-ca\") pod \"b55db003-49f0-437d-a303-0f3d39fe9e68\" (UID: \"b55db003-49f0-437d-a303-0f3d39fe9e68\") "
Jul 15 17:34:49.244619 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:49.244583    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b55db003-49f0-437d-a303-0f3d39fe9e68-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b55db003-49f0-437d-a303-0f3d39fe9e68" (UID: "b55db003-49f0-437d-a303-0f3d39fe9e68"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:34:49.246123 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:49.246099    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b55db003-49f0-437d-a303-0f3d39fe9e68-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b55db003-49f0-437d-a303-0f3d39fe9e68" (UID: "b55db003-49f0-437d-a303-0f3d39fe9e68"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:34:49.246222 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:49.246156    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b55db003-49f0-437d-a303-0f3d39fe9e68-kube-api-access-2fwch" (OuterVolumeSpecName: "kube-api-access-2fwch") pod "b55db003-49f0-437d-a303-0f3d39fe9e68" (UID: "b55db003-49f0-437d-a303-0f3d39fe9e68"). InnerVolumeSpecName "kube-api-access-2fwch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:34:49.345420 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:49.345388    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b55db003-49f0-437d-a303-0f3d39fe9e68-openshift-service-ca\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:34:49.345420 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:49.345416    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55db003-49f0-437d-a303-0f3d39fe9e68-maas-api-tls\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:34:49.345420 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:49.345425    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fwch\" (UniqueName: \"kubernetes.io/projected/b55db003-49f0-437d-a303-0f3d39fe9e68-kube-api-access-2fwch\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:34:49.852749 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:49.852714    2575 generic.go:356] "Generic (PLEG): container finished" podID="b55db003-49f0-437d-a303-0f3d39fe9e68" containerID="ae353ce80e79af8981bfec5269e2c78ae1ba28ec16ca81bc0d7da77c2f2ebc0d" exitCode=0
Jul 15 17:34:49.852911 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:49.852753    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld" event={"ID":"b55db003-49f0-437d-a303-0f3d39fe9e68","Type":"ContainerDied","Data":"ae353ce80e79af8981bfec5269e2c78ae1ba28ec16ca81bc0d7da77c2f2ebc0d"}
Jul 15 17:34:49.852911 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:49.852769    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld"
Jul 15 17:34:49.852911 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:49.852793    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld" event={"ID":"b55db003-49f0-437d-a303-0f3d39fe9e68","Type":"ContainerDied","Data":"8803ccfc396c379f97f4d29f43afc4b94f92ba9598759aa3c6aa2b883472d72f"}
Jul 15 17:34:49.852911 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:49.852814    2575 scope.go:117] "RemoveContainer" containerID="ae353ce80e79af8981bfec5269e2c78ae1ba28ec16ca81bc0d7da77c2f2ebc0d"
Jul 15 17:34:49.862005 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:49.861982    2575 scope.go:117] "RemoveContainer" containerID="ae353ce80e79af8981bfec5269e2c78ae1ba28ec16ca81bc0d7da77c2f2ebc0d"
Jul 15 17:34:49.862263 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:34:49.862243    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae353ce80e79af8981bfec5269e2c78ae1ba28ec16ca81bc0d7da77c2f2ebc0d\": container with ID starting with ae353ce80e79af8981bfec5269e2c78ae1ba28ec16ca81bc0d7da77c2f2ebc0d not found: ID does not exist" containerID="ae353ce80e79af8981bfec5269e2c78ae1ba28ec16ca81bc0d7da77c2f2ebc0d"
Jul 15 17:34:49.862326 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:49.862278    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae353ce80e79af8981bfec5269e2c78ae1ba28ec16ca81bc0d7da77c2f2ebc0d"} err="failed to get container status \"ae353ce80e79af8981bfec5269e2c78ae1ba28ec16ca81bc0d7da77c2f2ebc0d\": rpc error: code = NotFound desc = could not find container \"ae353ce80e79af8981bfec5269e2c78ae1ba28ec16ca81bc0d7da77c2f2ebc0d\": container with ID starting with ae353ce80e79af8981bfec5269e2c78ae1ba28ec16ca81bc0d7da77c2f2ebc0d not found: ID does not exist"
Jul 15 17:34:49.874677 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:49.874653    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld"]
Jul 15 17:34:49.877611 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:49.877592    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-427b7c64-685687ff65-qq2ld"]
Jul 15 17:34:50.525390 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:50.525361    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b55db003-49f0-437d-a303-0f3d39fe9e68" path="/var/lib/kubelet/pods/b55db003-49f0-437d-a303-0f3d39fe9e68/volumes"
Jul 15 17:34:58.534132 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:34:58.534102    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"]
Jul 15 17:35:13.554339 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:13.554291    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl" podUID="772b7ac0-f909-415a-8642-5976ce3e6d5e" containerName="main" containerID="cri-o://8ee0e8dde8138a3b24b9248e1fcc453bb44ad927cadf81fac2cd19a4e67cbc08" gracePeriod=45
Jul 15 17:35:13.801013 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:13.800987    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:35:13.971331 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:13.971302    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-home\") pod \"772b7ac0-f909-415a-8642-5976ce3e6d5e\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") "
Jul 15 17:35:13.971533 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:13.971381    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-model-cache\") pod \"772b7ac0-f909-415a-8642-5976ce3e6d5e\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") "
Jul 15 17:35:13.971533 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:13.971411    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/772b7ac0-f909-415a-8642-5976ce3e6d5e-tls-certs\") pod \"772b7ac0-f909-415a-8642-5976ce3e6d5e\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") "
Jul 15 17:35:13.971533 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:13.971439    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhjps\" (UniqueName: \"kubernetes.io/projected/772b7ac0-f909-415a-8642-5976ce3e6d5e-kube-api-access-bhjps\") pod \"772b7ac0-f909-415a-8642-5976ce3e6d5e\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") "
Jul 15 17:35:13.971533 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:13.971467    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-dshm\") pod \"772b7ac0-f909-415a-8642-5976ce3e6d5e\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") "
Jul 15 17:35:13.971533 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:13.971511    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-tmp-dir\") pod \"772b7ac0-f909-415a-8642-5976ce3e6d5e\" (UID: \"772b7ac0-f909-415a-8642-5976ce3e6d5e\") "
Jul 15 17:35:13.971860 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:13.971683    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-model-cache" (OuterVolumeSpecName: "model-cache") pod "772b7ac0-f909-415a-8642-5976ce3e6d5e" (UID: "772b7ac0-f909-415a-8642-5976ce3e6d5e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:35:13.971924 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:13.971876    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "772b7ac0-f909-415a-8642-5976ce3e6d5e" (UID: "772b7ac0-f909-415a-8642-5976ce3e6d5e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:35:13.971924 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:13.971913    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-model-cache\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:35:13.972012 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:13.971923    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-home" (OuterVolumeSpecName: "home") pod "772b7ac0-f909-415a-8642-5976ce3e6d5e" (UID: "772b7ac0-f909-415a-8642-5976ce3e6d5e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:35:13.973763 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:13.973733    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-dshm" (OuterVolumeSpecName: "dshm") pod "772b7ac0-f909-415a-8642-5976ce3e6d5e" (UID: "772b7ac0-f909-415a-8642-5976ce3e6d5e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:35:13.973977 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:13.973957    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/772b7ac0-f909-415a-8642-5976ce3e6d5e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "772b7ac0-f909-415a-8642-5976ce3e6d5e" (UID: "772b7ac0-f909-415a-8642-5976ce3e6d5e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:35:13.974047 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:13.973975    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/772b7ac0-f909-415a-8642-5976ce3e6d5e-kube-api-access-bhjps" (OuterVolumeSpecName: "kube-api-access-bhjps") pod "772b7ac0-f909-415a-8642-5976ce3e6d5e" (UID: "772b7ac0-f909-415a-8642-5976ce3e6d5e"). InnerVolumeSpecName "kube-api-access-bhjps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:35:13.977607 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:13.977578    2575 generic.go:356] "Generic (PLEG): container finished" podID="772b7ac0-f909-415a-8642-5976ce3e6d5e" containerID="8ee0e8dde8138a3b24b9248e1fcc453bb44ad927cadf81fac2cd19a4e67cbc08" exitCode=0
Jul 15 17:35:13.977703 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:13.977635    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"
Jul 15 17:35:13.977750 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:13.977632    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl" event={"ID":"772b7ac0-f909-415a-8642-5976ce3e6d5e","Type":"ContainerDied","Data":"8ee0e8dde8138a3b24b9248e1fcc453bb44ad927cadf81fac2cd19a4e67cbc08"}
Jul 15 17:35:13.977787 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:13.977742    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl" event={"ID":"772b7ac0-f909-415a-8642-5976ce3e6d5e","Type":"ContainerDied","Data":"aa6407f7cf43443f41945456f1248220eb2aa4aee4d136efc297eead5823870f"}
Jul 15 17:35:13.977787 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:13.977769    2575 scope.go:117] "RemoveContainer" containerID="8ee0e8dde8138a3b24b9248e1fcc453bb44ad927cadf81fac2cd19a4e67cbc08"
Jul 15 17:35:13.993024 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:13.993008    2575 scope.go:117] "RemoveContainer" containerID="8ee0e8dde8138a3b24b9248e1fcc453bb44ad927cadf81fac2cd19a4e67cbc08"
Jul 15 17:35:13.993307 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:35:13.993279    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ee0e8dde8138a3b24b9248e1fcc453bb44ad927cadf81fac2cd19a4e67cbc08\": container with ID starting with 8ee0e8dde8138a3b24b9248e1fcc453bb44ad927cadf81fac2cd19a4e67cbc08 not found: ID does not exist" containerID="8ee0e8dde8138a3b24b9248e1fcc453bb44ad927cadf81fac2cd19a4e67cbc08"
Jul 15 17:35:13.993393 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:13.993306    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ee0e8dde8138a3b24b9248e1fcc453bb44ad927cadf81fac2cd19a4e67cbc08"} err="failed to get container status \"8ee0e8dde8138a3b24b9248e1fcc453bb44ad927cadf81fac2cd19a4e67cbc08\": rpc error: code = NotFound desc = could not find container \"8ee0e8dde8138a3b24b9248e1fcc453bb44ad927cadf81fac2cd19a4e67cbc08\": container with ID starting with 8ee0e8dde8138a3b24b9248e1fcc453bb44ad927cadf81fac2cd19a4e67cbc08 not found: ID does not exist"
Jul 15 17:35:14.004205 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:14.004181    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"]
Jul 15 17:35:14.005895 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:14.005876    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-427b7c64/e2e-lifecycle-model-427b7c64-kserve-7c79948bcf-zkhpl"]
Jul 15 17:35:14.072718 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:14.072696    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-home\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:35:14.072822 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:14.072720    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/772b7ac0-f909-415a-8642-5976ce3e6d5e-tls-certs\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:35:14.072822 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:14.072736    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhjps\" (UniqueName: \"kubernetes.io/projected/772b7ac0-f909-415a-8642-5976ce3e6d5e-kube-api-access-bhjps\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:35:14.072822 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:14.072748    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-dshm\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:35:14.072822 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:14.072760    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/772b7ac0-f909-415a-8642-5976ce3e6d5e-tmp-dir\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:35:14.526004 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:14.525971    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="772b7ac0-f909-415a-8642-5976ce3e6d5e" path="/var/lib/kubelet/pods/772b7ac0-f909-415a-8642-5976ce3e6d5e/volumes"
Jul 15 17:35:45.893215 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:45.893178    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"]
Jul 15 17:35:45.893672 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:45.893393    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk" podUID="e0f951ae-739a-4281-9b58-c202914a5c67" containerName="istio-proxy" containerID="cri-o://cd00e7a1b24c1774d4d42710739bf4e02ddd8a9738d3ec39b06f5e42795d741c" gracePeriod=30
Jul 15 17:35:50.017753 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:50.017713    2575 patch_prober.go:28] interesting pod/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 17:35:50.018118 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:50.017770    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk" podUID="e0f951ae-739a-4281-9b58-c202914a5c67" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 17:35:51.144104 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.144075    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:35:51.160659 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.160592    2575 generic.go:356] "Generic (PLEG): container finished" podID="e0f951ae-739a-4281-9b58-c202914a5c67" containerID="cd00e7a1b24c1774d4d42710739bf4e02ddd8a9738d3ec39b06f5e42795d741c" exitCode=0
Jul 15 17:35:51.160659 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.160640    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk" event={"ID":"e0f951ae-739a-4281-9b58-c202914a5c67","Type":"ContainerDied","Data":"cd00e7a1b24c1774d4d42710739bf4e02ddd8a9738d3ec39b06f5e42795d741c"}
Jul 15 17:35:51.160659 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.160653    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"
Jul 15 17:35:51.160870 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.160685    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk" event={"ID":"e0f951ae-739a-4281-9b58-c202914a5c67","Type":"ContainerDied","Data":"ad505e4a4366b17070bdc31c14811f84456b0fe14c3e21e35cdd57a192443d05"}
Jul 15 17:35:51.160870 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.160707    2575 scope.go:117] "RemoveContainer" containerID="cd00e7a1b24c1774d4d42710739bf4e02ddd8a9738d3ec39b06f5e42795d741c"
Jul 15 17:35:51.171541 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.171516    2575 scope.go:117] "RemoveContainer" containerID="cbb60e4971de5ecb1b16796958b5bd76d6a68ddfab9508a8fe582efe35e979e2"
Jul 15 17:35:51.179980 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.179958    2575 scope.go:117] "RemoveContainer" containerID="cd00e7a1b24c1774d4d42710739bf4e02ddd8a9738d3ec39b06f5e42795d741c"
Jul 15 17:35:51.180289 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:35:51.180270    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd00e7a1b24c1774d4d42710739bf4e02ddd8a9738d3ec39b06f5e42795d741c\": container with ID starting with cd00e7a1b24c1774d4d42710739bf4e02ddd8a9738d3ec39b06f5e42795d741c not found: ID does not exist" containerID="cd00e7a1b24c1774d4d42710739bf4e02ddd8a9738d3ec39b06f5e42795d741c"
Jul 15 17:35:51.180344 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.180297    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd00e7a1b24c1774d4d42710739bf4e02ddd8a9738d3ec39b06f5e42795d741c"} err="failed to get container status \"cd00e7a1b24c1774d4d42710739bf4e02ddd8a9738d3ec39b06f5e42795d741c\": rpc error: code = NotFound desc = could not find container \"cd00e7a1b24c1774d4d42710739bf4e02ddd8a9738d3ec39b06f5e42795d741c\": container with ID starting with cd00e7a1b24c1774d4d42710739bf4e02ddd8a9738d3ec39b06f5e42795d741c not found: ID does not exist"
Jul 15 17:35:51.180344 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.180314    2575 scope.go:117] "RemoveContainer" containerID="cbb60e4971de5ecb1b16796958b5bd76d6a68ddfab9508a8fe582efe35e979e2"
Jul 15 17:35:51.180719 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:35:51.180550    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbb60e4971de5ecb1b16796958b5bd76d6a68ddfab9508a8fe582efe35e979e2\": container with ID starting with cbb60e4971de5ecb1b16796958b5bd76d6a68ddfab9508a8fe582efe35e979e2 not found: ID does not exist" containerID="cbb60e4971de5ecb1b16796958b5bd76d6a68ddfab9508a8fe582efe35e979e2"
Jul 15 17:35:51.180796 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.180727    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbb60e4971de5ecb1b16796958b5bd76d6a68ddfab9508a8fe582efe35e979e2"} err="failed to get container status \"cbb60e4971de5ecb1b16796958b5bd76d6a68ddfab9508a8fe582efe35e979e2\": rpc error: code = NotFound desc = could not find container \"cbb60e4971de5ecb1b16796958b5bd76d6a68ddfab9508a8fe582efe35e979e2\": container with ID starting with cbb60e4971de5ecb1b16796958b5bd76d6a68ddfab9508a8fe582efe35e979e2 not found: ID does not exist"
Jul 15 17:35:51.303082 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.303056    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0f951ae-739a-4281-9b58-c202914a5c67-istiod-ca-cert\") pod \"e0f951ae-739a-4281-9b58-c202914a5c67\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") "
Jul 15 17:35:51.303082 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.303087    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-workload-socket\") pod \"e0f951ae-739a-4281-9b58-c202914a5c67\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") "
Jul 15 17:35:51.303284 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.303149    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-credential-socket\") pod \"e0f951ae-739a-4281-9b58-c202914a5c67\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") "
Jul 15 17:35:51.303284 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.303239    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0f951ae-739a-4281-9b58-c202914a5c67-istio-token\") pod \"e0f951ae-739a-4281-9b58-c202914a5c67\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") "
Jul 15 17:35:51.303396 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.303305    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0f951ae-739a-4281-9b58-c202914a5c67-istio-podinfo\") pod \"e0f951ae-739a-4281-9b58-c202914a5c67\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") "
Jul 15 17:35:51.303396 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.303333    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-istio-data\") pod \"e0f951ae-739a-4281-9b58-c202914a5c67\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") "
Jul 15 17:35:51.303396 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.303368    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-workload-certs\") pod \"e0f951ae-739a-4281-9b58-c202914a5c67\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") "
Jul 15 17:35:51.303549 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.303399    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2k2h\" (UniqueName: \"kubernetes.io/projected/e0f951ae-739a-4281-9b58-c202914a5c67-kube-api-access-d2k2h\") pod \"e0f951ae-739a-4281-9b58-c202914a5c67\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") "
Jul 15 17:35:51.303549 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.303425    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e0f951ae-739a-4281-9b58-c202914a5c67" (UID: "e0f951ae-739a-4281-9b58-c202914a5c67"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:35:51.303549 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.303441    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e0f951ae-739a-4281-9b58-c202914a5c67" (UID: "e0f951ae-739a-4281-9b58-c202914a5c67"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:35:51.303549 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.303452    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0f951ae-739a-4281-9b58-c202914a5c67-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e0f951ae-739a-4281-9b58-c202914a5c67" (UID: "e0f951ae-739a-4281-9b58-c202914a5c67"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:35:51.303549 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.303452    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-istio-envoy\") pod \"e0f951ae-739a-4281-9b58-c202914a5c67\" (UID: \"e0f951ae-739a-4281-9b58-c202914a5c67\") "
Jul 15 17:35:51.303841 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.303720    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e0f951ae-739a-4281-9b58-c202914a5c67" (UID: "e0f951ae-739a-4281-9b58-c202914a5c67"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:35:51.303897 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.303850    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-workload-certs\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:35:51.303897 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.303869    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0f951ae-739a-4281-9b58-c202914a5c67-istiod-ca-cert\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:35:51.303897 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.303883    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-workload-socket\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:35:51.304053 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.303897    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-credential-socket\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:35:51.304372 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.304342    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-istio-data" (OuterVolumeSpecName: "istio-data") pod "e0f951ae-739a-4281-9b58-c202914a5c67" (UID: "e0f951ae-739a-4281-9b58-c202914a5c67"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:35:51.305546 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.305510    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0f951ae-739a-4281-9b58-c202914a5c67-istio-token" (OuterVolumeSpecName: "istio-token") pod "e0f951ae-739a-4281-9b58-c202914a5c67" (UID: "e0f951ae-739a-4281-9b58-c202914a5c67"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:35:51.305654 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.305599    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0f951ae-739a-4281-9b58-c202914a5c67-kube-api-access-d2k2h" (OuterVolumeSpecName: "kube-api-access-d2k2h") pod "e0f951ae-739a-4281-9b58-c202914a5c67" (UID: "e0f951ae-739a-4281-9b58-c202914a5c67"). InnerVolumeSpecName "kube-api-access-d2k2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:35:51.305654 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.305600    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e0f951ae-739a-4281-9b58-c202914a5c67-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e0f951ae-739a-4281-9b58-c202914a5c67" (UID: "e0f951ae-739a-4281-9b58-c202914a5c67"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 17:35:51.305654 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.305605    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e0f951ae-739a-4281-9b58-c202914a5c67" (UID: "e0f951ae-739a-4281-9b58-c202914a5c67"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:35:51.405151 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.405129    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0f951ae-739a-4281-9b58-c202914a5c67-istio-token\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:35:51.405151 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.405149    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0f951ae-739a-4281-9b58-c202914a5c67-istio-podinfo\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:35:51.405285 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.405158    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-istio-data\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:35:51.405285 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.405167    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2k2h\" (UniqueName: \"kubernetes.io/projected/e0f951ae-739a-4281-9b58-c202914a5c67-kube-api-access-d2k2h\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:35:51.405285 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.405176    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0f951ae-739a-4281-9b58-c202914a5c67-istio-envoy\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:35:51.482838 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.482812    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"]
Jul 15 17:35:51.486841 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:51.486821    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-427b7c64-openshift-default-6c9ff4ffcf-dqlzk"]
Jul 15 17:35:52.525757 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:35:52.525727    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0f951ae-739a-4281-9b58-c202914a5c67" path="/var/lib/kubelet/pods/e0f951ae-739a-4281-9b58-c202914a5c67/volumes"
Jul 15 17:36:24.567973 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:24.567943    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 17:36:24.574952 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:24.574931    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 17:36:41.285163 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.285126    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"]
Jul 15 17:36:41.285525 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.285508    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0f951ae-739a-4281-9b58-c202914a5c67" containerName="istio-proxy"
Jul 15 17:36:41.285525 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.285519    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0f951ae-739a-4281-9b58-c202914a5c67" containerName="istio-proxy"
Jul 15 17:36:41.285619 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.285530    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="772b7ac0-f909-415a-8642-5976ce3e6d5e" containerName="main"
Jul 15 17:36:41.285619 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.285536    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="772b7ac0-f909-415a-8642-5976ce3e6d5e" containerName="main"
Jul 15 17:36:41.285619 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.285546    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0f951ae-739a-4281-9b58-c202914a5c67" containerName="istio-proxy"
Jul 15 17:36:41.285619 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.285552    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0f951ae-739a-4281-9b58-c202914a5c67" containerName="istio-proxy"
Jul 15 17:36:41.285619 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.285573    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b55db003-49f0-437d-a303-0f3d39fe9e68" containerName="maas-api"
Jul 15 17:36:41.285619 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.285579    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b55db003-49f0-437d-a303-0f3d39fe9e68" containerName="maas-api"
Jul 15 17:36:41.285803 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.285645    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0f951ae-739a-4281-9b58-c202914a5c67" containerName="istio-proxy"
Jul 15 17:36:41.285803 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.285653    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b55db003-49f0-437d-a303-0f3d39fe9e68" containerName="maas-api"
Jul 15 17:36:41.285803 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.285659    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="772b7ac0-f909-415a-8642-5976ce3e6d5e" containerName="main"
Jul 15 17:36:41.285803 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.285665    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0f951ae-739a-4281-9b58-c202914a5c67" containerName="istio-proxy"
Jul 15 17:36:41.288618 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.288603    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.290751 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.290733    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-b525cf-openshift-default-dockercfg-xr9cg\""
Jul 15 17:36:41.298319 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.298295    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"]
Jul 15 17:36:41.422167 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.422134    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgsfq\" (UniqueName: \"kubernetes.io/projected/de8e5b20-89a3-4262-8e63-def07d1c9634-kube-api-access-zgsfq\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.422167 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.422170    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-workload-certs\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.422418 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.422200    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-podinfo\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.422418 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.422245    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de8e5b20-89a3-4262-8e63-def07d1c9634-istiod-ca-cert\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.422418 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.422312    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-data\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.422418 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.422367    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-envoy\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.422418 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.422417    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-token\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.422689 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.422461    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-credential-socket\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.422689 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.422488    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-workload-socket\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.523326 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.523291    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-token\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.523481 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.523331    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-credential-socket\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.523481 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.523355    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-workload-socket\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.523481 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.523390    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgsfq\" (UniqueName: \"kubernetes.io/projected/de8e5b20-89a3-4262-8e63-def07d1c9634-kube-api-access-zgsfq\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.523481 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.523415    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-workload-certs\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.523481 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.523455    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-podinfo\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.523481 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.523475    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de8e5b20-89a3-4262-8e63-def07d1c9634-istiod-ca-cert\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.523839 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.523538    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-data\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.523839 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.523648    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-envoy\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.523839 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.523759    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-credential-socket\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.523839 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.523801    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-workload-socket\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.523978 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.523942    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-workload-certs\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.524037 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.524014    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-data\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.524106 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.524090    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de8e5b20-89a3-4262-8e63-def07d1c9634-istiod-ca-cert\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.525977 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.525957    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-envoy\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.526091 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.526074    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-podinfo\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.531121 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.531099    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgsfq\" (UniqueName: \"kubernetes.io/projected/de8e5b20-89a3-4262-8e63-def07d1c9634-kube-api-access-zgsfq\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.531669 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.531648    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-token\") pod \"e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") " pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.600233 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.600171    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:41.731955 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.731907    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"]
Jul 15 17:36:41.734196 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:36:41.734168    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde8e5b20_89a3_4262_8e63_def07d1c9634.slice/crio-d7e32eafa1e98646c7979610d342461deb0f135d2dc23f49fb5b634bc6bf1500 WatchSource:0}: Error finding container d7e32eafa1e98646c7979610d342461deb0f135d2dc23f49fb5b634bc6bf1500: Status 404 returned error can't find the container with id d7e32eafa1e98646c7979610d342461deb0f135d2dc23f49fb5b634bc6bf1500
Jul 15 17:36:41.736235 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.736201    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:36:41.736291 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.736264    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:36:41.736324 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:41.736292    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:36:42.405734 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:42.405697    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57" event={"ID":"de8e5b20-89a3-4262-8e63-def07d1c9634","Type":"ContainerStarted","Data":"f0244fb2c6fc77e432c735e933967092fe11104a2e615e8ceb7e53f5b61a037f"}
Jul 15 17:36:42.406126 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:42.405758    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57" event={"ID":"de8e5b20-89a3-4262-8e63-def07d1c9634","Type":"ContainerStarted","Data":"d7e32eafa1e98646c7979610d342461deb0f135d2dc23f49fb5b634bc6bf1500"}
Jul 15 17:36:42.426261 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:42.426204    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57" podStartSLOduration=1.426186601 podStartE2EDuration="1.426186601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:36:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:36:42.425194246 +0000 UTC m=+2418.442607342" watchObservedRunningTime="2026-07-15 17:36:42.426186601 +0000 UTC m=+2418.443599699"
Jul 15 17:36:42.600514 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:42.600482    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:42.602054 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:42.602029    2575 patch_prober.go:28] interesting pod/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 15 17:36:42.602171 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:42.602077    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57" podUID="de8e5b20-89a3-4262-8e63-def07d1c9634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 15 17:36:43.605106 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:43.605077    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:44.416221 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:44.416187    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:44.417204 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:44.417181    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:36:53.205856 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:53.205822    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d"]
Jul 15 17:36:53.209280 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:53.209259    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d"
Jul 15 17:36:53.212495 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:53.212465    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-ed384a-serving-cert\""
Jul 15 17:36:53.221484 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:53.221451    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d"]
Jul 15 17:36:53.327281 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:53.327234    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87abafec-8da3-40bb-a396-fc0750c6445f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d\" (UID: \"87abafec-8da3-40bb-a396-fc0750c6445f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d"
Jul 15 17:36:53.327622 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:53.327596    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87abafec-8da3-40bb-a396-fc0750c6445f-maas-api-tls\") pod \"maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d\" (UID: \"87abafec-8da3-40bb-a396-fc0750c6445f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d"
Jul 15 17:36:53.327883 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:53.327682    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nw9x\" (UniqueName: \"kubernetes.io/projected/87abafec-8da3-40bb-a396-fc0750c6445f-kube-api-access-5nw9x\") pod \"maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d\" (UID: \"87abafec-8da3-40bb-a396-fc0750c6445f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d"
Jul 15 17:36:53.428645 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:53.428607    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87abafec-8da3-40bb-a396-fc0750c6445f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d\" (UID: \"87abafec-8da3-40bb-a396-fc0750c6445f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d"
Jul 15 17:36:53.428815 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:53.428671    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87abafec-8da3-40bb-a396-fc0750c6445f-maas-api-tls\") pod \"maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d\" (UID: \"87abafec-8da3-40bb-a396-fc0750c6445f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d"
Jul 15 17:36:53.428815 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:53.428707    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nw9x\" (UniqueName: \"kubernetes.io/projected/87abafec-8da3-40bb-a396-fc0750c6445f-kube-api-access-5nw9x\") pod \"maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d\" (UID: \"87abafec-8da3-40bb-a396-fc0750c6445f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d"
Jul 15 17:36:53.428901 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:36:53.428829    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-serving-cert: secret "maas-api-e2e-inf-b-ed384a-serving-cert" not found
Jul 15 17:36:53.428901 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:36:53.428891    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87abafec-8da3-40bb-a396-fc0750c6445f-maas-api-tls podName:87abafec-8da3-40bb-a396-fc0750c6445f nodeName:}" failed. No retries permitted until 2026-07-15 17:36:53.928875399 +0000 UTC m=+2429.946288474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/87abafec-8da3-40bb-a396-fc0750c6445f-maas-api-tls") pod "maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d" (UID: "87abafec-8da3-40bb-a396-fc0750c6445f") : secret "maas-api-e2e-inf-b-ed384a-serving-cert" not found
Jul 15 17:36:53.429314 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:53.429297    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87abafec-8da3-40bb-a396-fc0750c6445f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d\" (UID: \"87abafec-8da3-40bb-a396-fc0750c6445f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d"
Jul 15 17:36:53.444867 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:53.444834    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nw9x\" (UniqueName: \"kubernetes.io/projected/87abafec-8da3-40bb-a396-fc0750c6445f-kube-api-access-5nw9x\") pod \"maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d\" (UID: \"87abafec-8da3-40bb-a396-fc0750c6445f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d"
Jul 15 17:36:53.934325 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:53.934295    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87abafec-8da3-40bb-a396-fc0750c6445f-maas-api-tls\") pod \"maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d\" (UID: \"87abafec-8da3-40bb-a396-fc0750c6445f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d"
Jul 15 17:36:53.936741 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:53.936712    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87abafec-8da3-40bb-a396-fc0750c6445f-maas-api-tls\") pod \"maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d\" (UID: \"87abafec-8da3-40bb-a396-fc0750c6445f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d"
Jul 15 17:36:54.120977 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:54.120948    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d"
Jul 15 17:36:54.313048 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:54.312984    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d"]
Jul 15 17:36:54.315495 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:36:54.315468    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87abafec_8da3_40bb_a396_fc0750c6445f.slice/crio-ff2f6abf7256949d019c78a44eea930aab0865b1c0b823d4e86f13de976a8587 WatchSource:0}: Error finding container ff2f6abf7256949d019c78a44eea930aab0865b1c0b823d4e86f13de976a8587: Status 404 returned error can't find the container with id ff2f6abf7256949d019c78a44eea930aab0865b1c0b823d4e86f13de976a8587
Jul 15 17:36:54.472843 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:54.472749    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d" event={"ID":"87abafec-8da3-40bb-a396-fc0750c6445f","Type":"ContainerStarted","Data":"ff2f6abf7256949d019c78a44eea930aab0865b1c0b823d4e86f13de976a8587"}
Jul 15 17:36:55.478985 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:55.478952    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d" event={"ID":"87abafec-8da3-40bb-a396-fc0750c6445f","Type":"ContainerStarted","Data":"5c7a3e2b61a35e2390d01512b135aaf35a54d723ebeb8f89fca51cec470f1621"}
Jul 15 17:36:55.479455 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:55.479079    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d"
Jul 15 17:36:55.504725 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:36:55.504678    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d" podStartSLOduration=1.823521919 podStartE2EDuration="2.504662151s" totalImagesPullingTime="681.140232ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:36:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:36:55.502961891 +0000 UTC m=+2431.520374987" watchObservedRunningTime="2026-07-15 17:36:55.504662151 +0000 UTC m=+2431.522075247"
Jul 15 17:37:00.458703 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:00.458666    2575 patch_prober.go:28] interesting pod/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 15 17:37:00.459096 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:00.458718    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57" podUID="de8e5b20-89a3-4262-8e63-def07d1c9634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 15 17:37:01.490422 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:01.490395    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d"
Jul 15 17:37:01.509722 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:01.509695    2575 generic.go:356] "Generic (PLEG): container finished" podID="de8e5b20-89a3-4262-8e63-def07d1c9634" containerID="f0244fb2c6fc77e432c735e933967092fe11104a2e615e8ceb7e53f5b61a037f" exitCode=0
Jul 15 17:37:01.509871 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:01.509769    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57" event={"ID":"de8e5b20-89a3-4262-8e63-def07d1c9634","Type":"ContainerDied","Data":"f0244fb2c6fc77e432c735e933967092fe11104a2e615e8ceb7e53f5b61a037f"}
Jul 15 17:37:01.510106 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:01.510092    2575 scope.go:117] "RemoveContainer" containerID="f0244fb2c6fc77e432c735e933967092fe11104a2e615e8ceb7e53f5b61a037f"
Jul 15 17:37:01.511028 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:01.510999    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:37:01.511159 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:01.511076    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:37:01.511159 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:01.511126    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:37:02.516624 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:02.516592    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57" event={"ID":"de8e5b20-89a3-4262-8e63-def07d1c9634","Type":"ContainerStarted","Data":"c2b45f9310297b5790ddc4eb40560022e09590c25d428f5516d1166937e56e5e"}
Jul 15 17:37:02.601291 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:02.601256    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:37:02.601638 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:02.601608    2575 patch_prober.go:28] interesting pod/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 15 17:37:02.601757 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:02.601659    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57" podUID="de8e5b20-89a3-4262-8e63-def07d1c9634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 15 17:37:03.600783 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:03.600750    2575 patch_prober.go:28] interesting pod/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 15 17:37:03.601173 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:03.600807    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57" podUID="de8e5b20-89a3-4262-8e63-def07d1c9634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 15 17:37:04.601283 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:04.601238    2575 patch_prober.go:28] interesting pod/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 15 17:37:04.601698 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:04.601305    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57" podUID="de8e5b20-89a3-4262-8e63-def07d1c9634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 15 17:37:05.601090 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:05.601057    2575 patch_prober.go:28] interesting pod/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 15 17:37:05.601262 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:05.601108    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57" podUID="de8e5b20-89a3-4262-8e63-def07d1c9634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 15 17:37:06.604040 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:06.604009    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:37:06.604503 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:06.604289    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:37:06.604920 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:06.604905    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:37:08.939187 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:08.939154    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"]
Jul 15 17:37:08.942856 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:08.942837    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:08.947648 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:08.947624    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-b525cf\"/\"default-dockercfg-gft4z\""
Jul 15 17:37:08.947773 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:08.947654    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-b525cf\"/\"openshift-service-ca.crt\""
Jul 15 17:37:08.947773 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:08.947654    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-b525cf\"/\"test-model-b525cf-kserve-self-signed-certs\""
Jul 15 17:37:08.947881 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:08.947766    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-b525cf\"/\"kube-root-ca.crt\""
Jul 15 17:37:08.959893 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:08.959866    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"]
Jul 15 17:37:08.974412 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:08.974387    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-model-cache\") pod \"test-model-b525cf-kserve-6999549467-ngq64\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") " pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:08.974582 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:08.974427    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-tls-certs\") pod \"test-model-b525cf-kserve-6999549467-ngq64\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") " pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:08.974582 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:08.974481    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-home\") pod \"test-model-b525cf-kserve-6999549467-ngq64\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") " pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:08.974582 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:08.974499    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-dshm\") pod \"test-model-b525cf-kserve-6999549467-ngq64\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") " pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:08.974582 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:08.974539    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zcrb\" (UniqueName: \"kubernetes.io/projected/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-kube-api-access-4zcrb\") pod \"test-model-b525cf-kserve-6999549467-ngq64\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") " pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:08.974731 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:08.974599    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-tmp-dir\") pod \"test-model-b525cf-kserve-6999549467-ngq64\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") " pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:09.075550 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:09.075516    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-home\") pod \"test-model-b525cf-kserve-6999549467-ngq64\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") " pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:09.075725 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:09.075584    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-dshm\") pod \"test-model-b525cf-kserve-6999549467-ngq64\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") " pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:09.075725 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:09.075613    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zcrb\" (UniqueName: \"kubernetes.io/projected/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-kube-api-access-4zcrb\") pod \"test-model-b525cf-kserve-6999549467-ngq64\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") " pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:09.075725 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:09.075670    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-tmp-dir\") pod \"test-model-b525cf-kserve-6999549467-ngq64\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") " pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:09.075889 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:09.075728    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-model-cache\") pod \"test-model-b525cf-kserve-6999549467-ngq64\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") " pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:09.075889 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:09.075769    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-tls-certs\") pod \"test-model-b525cf-kserve-6999549467-ngq64\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") " pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:09.075994 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:09.075955    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-home\") pod \"test-model-b525cf-kserve-6999549467-ngq64\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") " pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:09.076060 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:09.076041    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-tmp-dir\") pod \"test-model-b525cf-kserve-6999549467-ngq64\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") " pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:09.076124 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:09.076081    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-model-cache\") pod \"test-model-b525cf-kserve-6999549467-ngq64\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") " pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:09.077699 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:09.077680    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-dshm\") pod \"test-model-b525cf-kserve-6999549467-ngq64\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") " pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:09.078170 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:09.078152    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-tls-certs\") pod \"test-model-b525cf-kserve-6999549467-ngq64\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") " pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:09.087283 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:09.087258    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zcrb\" (UniqueName: \"kubernetes.io/projected/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-kube-api-access-4zcrb\") pod \"test-model-b525cf-kserve-6999549467-ngq64\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") " pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:09.252975 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:09.252894    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:09.400336 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:09.400312    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"]
Jul 15 17:37:09.402246 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:37:09.402219    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7dc4fbcb_8753_44fe_86eb_fd99d8eac40e.slice/crio-094e12382448abcfa23b3275d5bf5fd88bc6103c00901045e0a8db4275791d7d WatchSource:0}: Error finding container 094e12382448abcfa23b3275d5bf5fd88bc6103c00901045e0a8db4275791d7d: Status 404 returned error can't find the container with id 094e12382448abcfa23b3275d5bf5fd88bc6103c00901045e0a8db4275791d7d
Jul 15 17:37:09.552592 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:09.552488    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64" event={"ID":"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e","Type":"ContainerStarted","Data":"974b71bb7f4609ec8fbb95468889b7a1e44f376607d8f67dffdeef8c180450b4"}
Jul 15 17:37:09.552592 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:09.552531    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64" event={"ID":"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e","Type":"ContainerStarted","Data":"094e12382448abcfa23b3275d5bf5fd88bc6103c00901045e0a8db4275791d7d"}
Jul 15 17:37:09.586766 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:09.586721    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64" podStartSLOduration=1.5867069310000002 podStartE2EDuration="1.586706931s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:37:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:37:09.585542354 +0000 UTC m=+2445.602955449" watchObservedRunningTime="2026-07-15 17:37:09.586706931 +0000 UTC m=+2445.604120080"
Jul 15 17:37:19.253533 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:19.253484    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:19.253533 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:19.253541    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:19.272190 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:19.272161    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:19.610057 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:19.609979    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:37:32.835929 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.835895    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"]
Jul 15 17:37:32.839348 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.839326    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:32.851248 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.851215    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-ed384a\"/\"kube-root-ca.crt\""
Jul 15 17:37:32.851248 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.851215    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-ed384a\"/\"openshift-service-ca.crt\""
Jul 15 17:37:32.851644 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.851624    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-ed384a\"/\"test-model-ed384a-kserve-self-signed-certs\""
Jul 15 17:37:32.851644 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.851624    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-ed384a\"/\"default-dockercfg-gknn5\""
Jul 15 17:37:32.880033 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.880002    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"]
Jul 15 17:37:32.887288 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.887262    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c9c9943-7381-4752-9f3c-8984f96ae2ca-tls-certs\") pod \"test-model-ed384a-kserve-84d467784c-k8zvh\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") " pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:32.887449 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.887322    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnf5m\" (UniqueName: \"kubernetes.io/projected/4c9c9943-7381-4752-9f3c-8984f96ae2ca-kube-api-access-xnf5m\") pod \"test-model-ed384a-kserve-84d467784c-k8zvh\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") " pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:32.887449 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.887414    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-tmp-dir\") pod \"test-model-ed384a-kserve-84d467784c-k8zvh\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") " pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:32.887581 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.887478    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-model-cache\") pod \"test-model-ed384a-kserve-84d467784c-k8zvh\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") " pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:32.887581 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.887526    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-home\") pod \"test-model-ed384a-kserve-84d467784c-k8zvh\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") " pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:32.887679 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.887619    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-dshm\") pod \"test-model-ed384a-kserve-84d467784c-k8zvh\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") " pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:32.988951 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.988916    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-dshm\") pod \"test-model-ed384a-kserve-84d467784c-k8zvh\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") " pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:32.989160 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.988971    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c9c9943-7381-4752-9f3c-8984f96ae2ca-tls-certs\") pod \"test-model-ed384a-kserve-84d467784c-k8zvh\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") " pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:32.989160 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.989001    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnf5m\" (UniqueName: \"kubernetes.io/projected/4c9c9943-7381-4752-9f3c-8984f96ae2ca-kube-api-access-xnf5m\") pod \"test-model-ed384a-kserve-84d467784c-k8zvh\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") " pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:32.989160 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.989027    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-tmp-dir\") pod \"test-model-ed384a-kserve-84d467784c-k8zvh\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") " pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:32.989160 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.989061    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-model-cache\") pod \"test-model-ed384a-kserve-84d467784c-k8zvh\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") " pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:32.989160 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.989096    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-home\") pod \"test-model-ed384a-kserve-84d467784c-k8zvh\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") " pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:32.989469 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.989432    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-tmp-dir\") pod \"test-model-ed384a-kserve-84d467784c-k8zvh\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") " pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:32.989626 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.989494    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-model-cache\") pod \"test-model-ed384a-kserve-84d467784c-k8zvh\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") " pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:32.989626 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.989509    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-home\") pod \"test-model-ed384a-kserve-84d467784c-k8zvh\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") " pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:32.991218 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.991198    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-dshm\") pod \"test-model-ed384a-kserve-84d467784c-k8zvh\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") " pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:32.991649 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:32.991628    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c9c9943-7381-4752-9f3c-8984f96ae2ca-tls-certs\") pod \"test-model-ed384a-kserve-84d467784c-k8zvh\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") " pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:33.001481 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:33.001452    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnf5m\" (UniqueName: \"kubernetes.io/projected/4c9c9943-7381-4752-9f3c-8984f96ae2ca-kube-api-access-xnf5m\") pod \"test-model-ed384a-kserve-84d467784c-k8zvh\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") " pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:33.148764 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:33.148717    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:33.292501 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:33.292477    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"]
Jul 15 17:37:33.294853 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:37:33.294823    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c9c9943_7381_4752_9f3c_8984f96ae2ca.slice/crio-207d69e05cefdb1bc25151182bb396825c0e9f5dc14632c5fe4e68547d0e2961 WatchSource:0}: Error finding container 207d69e05cefdb1bc25151182bb396825c0e9f5dc14632c5fe4e68547d0e2961: Status 404 returned error can't find the container with id 207d69e05cefdb1bc25151182bb396825c0e9f5dc14632c5fe4e68547d0e2961
Jul 15 17:37:33.669255 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:33.669222    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh" event={"ID":"4c9c9943-7381-4752-9f3c-8984f96ae2ca","Type":"ContainerStarted","Data":"25f11132817fa4eaf7d5133a6715e6d4a8be3530a6b8844e1c6b7951284efcca"}
Jul 15 17:37:33.669432 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:33.669265    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh" event={"ID":"4c9c9943-7381-4752-9f3c-8984f96ae2ca","Type":"ContainerStarted","Data":"207d69e05cefdb1bc25151182bb396825c0e9f5dc14632c5fe4e68547d0e2961"}
Jul 15 17:37:33.693061 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:33.693012    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh" podStartSLOduration=1.692997573 podStartE2EDuration="1.692997573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:37:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:37:33.690147237 +0000 UTC m=+2469.707560371" watchObservedRunningTime="2026-07-15 17:37:33.692997573 +0000 UTC m=+2469.710410670"
Jul 15 17:37:43.149141 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:43.149108    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:43.149554 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:43.149321    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:43.161485 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:43.161453    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:43.729662 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:43.729633    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:37:51.801136 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:51.801105    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"]
Jul 15 17:37:52.880493 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:37:52.880458    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"]
Jul 15 17:38:06.820804 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:06.820761    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64" podUID="7dc4fbcb-8753-44fe-86eb-fd99d8eac40e" containerName="main" containerID="cri-o://974b71bb7f4609ec8fbb95468889b7a1e44f376607d8f67dffdeef8c180450b4" gracePeriod=45
Jul 15 17:38:07.078645 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.078592    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:38:07.197679 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.197649    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-model-cache\") pod \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") "
Jul 15 17:38:07.197863 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.197716    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-home\") pod \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") "
Jul 15 17:38:07.197863 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.197732    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-dshm\") pod \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") "
Jul 15 17:38:07.197863 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.197747    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-tmp-dir\") pod \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") "
Jul 15 17:38:07.197863 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.197768    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-tls-certs\") pod \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") "
Jul 15 17:38:07.197863 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.197811    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zcrb\" (UniqueName: \"kubernetes.io/projected/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-kube-api-access-4zcrb\") pod \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\" (UID: \"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e\") "
Jul 15 17:38:07.198137 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.197958    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-model-cache" (OuterVolumeSpecName: "model-cache") pod "7dc4fbcb-8753-44fe-86eb-fd99d8eac40e" (UID: "7dc4fbcb-8753-44fe-86eb-fd99d8eac40e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:38:07.198137 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.198019    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7dc4fbcb-8753-44fe-86eb-fd99d8eac40e" (UID: "7dc4fbcb-8753-44fe-86eb-fd99d8eac40e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:38:07.198137 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.197962    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-home" (OuterVolumeSpecName: "home") pod "7dc4fbcb-8753-44fe-86eb-fd99d8eac40e" (UID: "7dc4fbcb-8753-44fe-86eb-fd99d8eac40e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:38:07.198137 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.198096    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-tmp-dir\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:07.198137 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.198117    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-model-cache\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:07.199916 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.199890    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7dc4fbcb-8753-44fe-86eb-fd99d8eac40e" (UID: "7dc4fbcb-8753-44fe-86eb-fd99d8eac40e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:38:07.200026 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.199929    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-dshm" (OuterVolumeSpecName: "dshm") pod "7dc4fbcb-8753-44fe-86eb-fd99d8eac40e" (UID: "7dc4fbcb-8753-44fe-86eb-fd99d8eac40e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:38:07.200026 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.199982    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-kube-api-access-4zcrb" (OuterVolumeSpecName: "kube-api-access-4zcrb") pod "7dc4fbcb-8753-44fe-86eb-fd99d8eac40e" (UID: "7dc4fbcb-8753-44fe-86eb-fd99d8eac40e"). InnerVolumeSpecName "kube-api-access-4zcrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:38:07.298905 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.298876    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zcrb\" (UniqueName: \"kubernetes.io/projected/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-kube-api-access-4zcrb\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:07.298905 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.298902    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-home\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:07.298905 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.298911    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-dshm\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:07.299121 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.298920    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e-tls-certs\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:07.846085 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.846052    2575 generic.go:356] "Generic (PLEG): container finished" podID="7dc4fbcb-8753-44fe-86eb-fd99d8eac40e" containerID="974b71bb7f4609ec8fbb95468889b7a1e44f376607d8f67dffdeef8c180450b4" exitCode=0
Jul 15 17:38:07.846542 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.846099    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"
Jul 15 17:38:07.846542 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.846121    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64" event={"ID":"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e","Type":"ContainerDied","Data":"974b71bb7f4609ec8fbb95468889b7a1e44f376607d8f67dffdeef8c180450b4"}
Jul 15 17:38:07.846542 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.846151    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64" event={"ID":"7dc4fbcb-8753-44fe-86eb-fd99d8eac40e","Type":"ContainerDied","Data":"094e12382448abcfa23b3275d5bf5fd88bc6103c00901045e0a8db4275791d7d"}
Jul 15 17:38:07.846542 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.846168    2575 scope.go:117] "RemoveContainer" containerID="974b71bb7f4609ec8fbb95468889b7a1e44f376607d8f67dffdeef8c180450b4"
Jul 15 17:38:07.855762 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.855746    2575 scope.go:117] "RemoveContainer" containerID="974b71bb7f4609ec8fbb95468889b7a1e44f376607d8f67dffdeef8c180450b4"
Jul 15 17:38:07.856008 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:38:07.855989    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"974b71bb7f4609ec8fbb95468889b7a1e44f376607d8f67dffdeef8c180450b4\": container with ID starting with 974b71bb7f4609ec8fbb95468889b7a1e44f376607d8f67dffdeef8c180450b4 not found: ID does not exist" containerID="974b71bb7f4609ec8fbb95468889b7a1e44f376607d8f67dffdeef8c180450b4"
Jul 15 17:38:07.856060 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.856017    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"974b71bb7f4609ec8fbb95468889b7a1e44f376607d8f67dffdeef8c180450b4"} err="failed to get container status \"974b71bb7f4609ec8fbb95468889b7a1e44f376607d8f67dffdeef8c180450b4\": rpc error: code = NotFound desc = could not find container \"974b71bb7f4609ec8fbb95468889b7a1e44f376607d8f67dffdeef8c180450b4\": container with ID starting with 974b71bb7f4609ec8fbb95468889b7a1e44f376607d8f67dffdeef8c180450b4 not found: ID does not exist"
Jul 15 17:38:07.876817 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.876794    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"]
Jul 15 17:38:07.885874 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.885835    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-b525cf/test-model-b525cf-kserve-6999549467-ngq64"]
Jul 15 17:38:07.900332 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:07.900302    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh" podUID="4c9c9943-7381-4752-9f3c-8984f96ae2ca" containerName="main" containerID="cri-o://25f11132817fa4eaf7d5133a6715e6d4a8be3530a6b8844e1c6b7951284efcca" gracePeriod=45
Jul 15 17:38:08.159534 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.159510    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:38:08.208178 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.208154    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-tmp-dir\") pod \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") "
Jul 15 17:38:08.208280 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.208187    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c9c9943-7381-4752-9f3c-8984f96ae2ca-tls-certs\") pod \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") "
Jul 15 17:38:08.208280 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.208215    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-dshm\") pod \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") "
Jul 15 17:38:08.208280 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.208246    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnf5m\" (UniqueName: \"kubernetes.io/projected/4c9c9943-7381-4752-9f3c-8984f96ae2ca-kube-api-access-xnf5m\") pod \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") "
Jul 15 17:38:08.208450 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.208311    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-model-cache\") pod \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") "
Jul 15 17:38:08.208450 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.208336    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-home\") pod \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\" (UID: \"4c9c9943-7381-4752-9f3c-8984f96ae2ca\") "
Jul 15 17:38:08.208554 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.208446    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4c9c9943-7381-4752-9f3c-8984f96ae2ca" (UID: "4c9c9943-7381-4752-9f3c-8984f96ae2ca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:38:08.208645 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.208588    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-model-cache" (OuterVolumeSpecName: "model-cache") pod "4c9c9943-7381-4752-9f3c-8984f96ae2ca" (UID: "4c9c9943-7381-4752-9f3c-8984f96ae2ca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:38:08.208645 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.208601    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-home" (OuterVolumeSpecName: "home") pod "4c9c9943-7381-4752-9f3c-8984f96ae2ca" (UID: "4c9c9943-7381-4752-9f3c-8984f96ae2ca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:38:08.208736 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.208713    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-model-cache\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:08.208736 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.208732    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-home\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:08.208803 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.208743    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-tmp-dir\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:08.210348 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.210329    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-dshm" (OuterVolumeSpecName: "dshm") pod "4c9c9943-7381-4752-9f3c-8984f96ae2ca" (UID: "4c9c9943-7381-4752-9f3c-8984f96ae2ca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:38:08.210702 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.210681    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c9c9943-7381-4752-9f3c-8984f96ae2ca-kube-api-access-xnf5m" (OuterVolumeSpecName: "kube-api-access-xnf5m") pod "4c9c9943-7381-4752-9f3c-8984f96ae2ca" (UID: "4c9c9943-7381-4752-9f3c-8984f96ae2ca"). InnerVolumeSpecName "kube-api-access-xnf5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:38:08.210702 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.210698    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c9c9943-7381-4752-9f3c-8984f96ae2ca-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4c9c9943-7381-4752-9f3c-8984f96ae2ca" (UID: "4c9c9943-7381-4752-9f3c-8984f96ae2ca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:38:08.309294 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.309270    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnf5m\" (UniqueName: \"kubernetes.io/projected/4c9c9943-7381-4752-9f3c-8984f96ae2ca-kube-api-access-xnf5m\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:08.309294 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.309295    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c9c9943-7381-4752-9f3c-8984f96ae2ca-tls-certs\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:08.309461 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.309305    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c9c9943-7381-4752-9f3c-8984f96ae2ca-dshm\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:08.526749 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.526675    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dc4fbcb-8753-44fe-86eb-fd99d8eac40e" path="/var/lib/kubelet/pods/7dc4fbcb-8753-44fe-86eb-fd99d8eac40e/volumes"
Jul 15 17:38:08.852413 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.852330    2575 generic.go:356] "Generic (PLEG): container finished" podID="4c9c9943-7381-4752-9f3c-8984f96ae2ca" containerID="25f11132817fa4eaf7d5133a6715e6d4a8be3530a6b8844e1c6b7951284efcca" exitCode=0
Jul 15 17:38:08.852413 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.852372    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"
Jul 15 17:38:08.852946 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.852416    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh" event={"ID":"4c9c9943-7381-4752-9f3c-8984f96ae2ca","Type":"ContainerDied","Data":"25f11132817fa4eaf7d5133a6715e6d4a8be3530a6b8844e1c6b7951284efcca"}
Jul 15 17:38:08.852946 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.852456    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh" event={"ID":"4c9c9943-7381-4752-9f3c-8984f96ae2ca","Type":"ContainerDied","Data":"207d69e05cefdb1bc25151182bb396825c0e9f5dc14632c5fe4e68547d0e2961"}
Jul 15 17:38:08.852946 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.852476    2575 scope.go:117] "RemoveContainer" containerID="25f11132817fa4eaf7d5133a6715e6d4a8be3530a6b8844e1c6b7951284efcca"
Jul 15 17:38:08.860944 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.860929    2575 scope.go:117] "RemoveContainer" containerID="25f11132817fa4eaf7d5133a6715e6d4a8be3530a6b8844e1c6b7951284efcca"
Jul 15 17:38:08.861208 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:38:08.861191    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25f11132817fa4eaf7d5133a6715e6d4a8be3530a6b8844e1c6b7951284efcca\": container with ID starting with 25f11132817fa4eaf7d5133a6715e6d4a8be3530a6b8844e1c6b7951284efcca not found: ID does not exist" containerID="25f11132817fa4eaf7d5133a6715e6d4a8be3530a6b8844e1c6b7951284efcca"
Jul 15 17:38:08.861259 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.861216    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25f11132817fa4eaf7d5133a6715e6d4a8be3530a6b8844e1c6b7951284efcca"} err="failed to get container status \"25f11132817fa4eaf7d5133a6715e6d4a8be3530a6b8844e1c6b7951284efcca\": rpc error: code = NotFound desc = could not find container \"25f11132817fa4eaf7d5133a6715e6d4a8be3530a6b8844e1c6b7951284efcca\": container with ID starting with 25f11132817fa4eaf7d5133a6715e6d4a8be3530a6b8844e1c6b7951284efcca not found: ID does not exist"
Jul 15 17:38:08.873995 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.873971    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"]
Jul 15 17:38:08.881750 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:08.881723    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-ed384a/test-model-ed384a-kserve-84d467784c-k8zvh"]
Jul 15 17:38:10.525780 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:10.525750    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c9c9943-7381-4752-9f3c-8984f96ae2ca" path="/var/lib/kubelet/pods/4c9c9943-7381-4752-9f3c-8984f96ae2ca/volumes"
Jul 15 17:38:19.281265 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:19.281184    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"]
Jul 15 17:38:19.281669 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:19.281428    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57" podUID="de8e5b20-89a3-4262-8e63-def07d1c9634" containerName="istio-proxy" containerID="cri-o://c2b45f9310297b5790ddc4eb40560022e09590c25d428f5516d1166937e56e5e" gracePeriod=30
Jul 15 17:38:21.606789 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:21.606754    2575 patch_prober.go:28] interesting pod/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 17:38:21.607180 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:21.606827    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57" podUID="de8e5b20-89a3-4262-8e63-def07d1c9634" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 17:38:24.538150 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.538124    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:38:24.657825 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.657795    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-workload-certs\") pod \"de8e5b20-89a3-4262-8e63-def07d1c9634\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") "
Jul 15 17:38:24.658021 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.657868    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de8e5b20-89a3-4262-8e63-def07d1c9634-istiod-ca-cert\") pod \"de8e5b20-89a3-4262-8e63-def07d1c9634\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") "
Jul 15 17:38:24.658021 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.657911    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-podinfo\") pod \"de8e5b20-89a3-4262-8e63-def07d1c9634\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") "
Jul 15 17:38:24.658021 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.657927    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-data\") pod \"de8e5b20-89a3-4262-8e63-def07d1c9634\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") "
Jul 15 17:38:24.658021 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.657949    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-workload-socket\") pod \"de8e5b20-89a3-4262-8e63-def07d1c9634\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") "
Jul 15 17:38:24.658242 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.658031    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-envoy\") pod \"de8e5b20-89a3-4262-8e63-def07d1c9634\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") "
Jul 15 17:38:24.658242 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.658079    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-credential-socket\") pod \"de8e5b20-89a3-4262-8e63-def07d1c9634\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") "
Jul 15 17:38:24.658242 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.658118    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgsfq\" (UniqueName: \"kubernetes.io/projected/de8e5b20-89a3-4262-8e63-def07d1c9634-kube-api-access-zgsfq\") pod \"de8e5b20-89a3-4262-8e63-def07d1c9634\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") "
Jul 15 17:38:24.658242 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.658157    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-token\") pod \"de8e5b20-89a3-4262-8e63-def07d1c9634\" (UID: \"de8e5b20-89a3-4262-8e63-def07d1c9634\") "
Jul 15 17:38:24.658242 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.658161    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "de8e5b20-89a3-4262-8e63-def07d1c9634" (UID: "de8e5b20-89a3-4262-8e63-def07d1c9634"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:38:24.658499 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.658359    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de8e5b20-89a3-4262-8e63-def07d1c9634-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "de8e5b20-89a3-4262-8e63-def07d1c9634" (UID: "de8e5b20-89a3-4262-8e63-def07d1c9634"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:38:24.658499 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.658459    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "de8e5b20-89a3-4262-8e63-def07d1c9634" (UID: "de8e5b20-89a3-4262-8e63-def07d1c9634"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:38:24.658713 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.658599    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "de8e5b20-89a3-4262-8e63-def07d1c9634" (UID: "de8e5b20-89a3-4262-8e63-def07d1c9634"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:38:24.658713 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.658663    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de8e5b20-89a3-4262-8e63-def07d1c9634-istiod-ca-cert\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:24.658713 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.658682    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-credential-socket\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:24.658713 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.658697    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-workload-certs\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:24.659047 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.659021    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-data" (OuterVolumeSpecName: "istio-data") pod "de8e5b20-89a3-4262-8e63-def07d1c9634" (UID: "de8e5b20-89a3-4262-8e63-def07d1c9634"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:38:24.660374 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.660350    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de8e5b20-89a3-4262-8e63-def07d1c9634-kube-api-access-zgsfq" (OuterVolumeSpecName: "kube-api-access-zgsfq") pod "de8e5b20-89a3-4262-8e63-def07d1c9634" (UID: "de8e5b20-89a3-4262-8e63-def07d1c9634"). InnerVolumeSpecName "kube-api-access-zgsfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:38:24.660496 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.660445    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "de8e5b20-89a3-4262-8e63-def07d1c9634" (UID: "de8e5b20-89a3-4262-8e63-def07d1c9634"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:38:24.660696 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.660676    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "de8e5b20-89a3-4262-8e63-def07d1c9634" (UID: "de8e5b20-89a3-4262-8e63-def07d1c9634"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 17:38:24.660828 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.660810    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-token" (OuterVolumeSpecName: "istio-token") pod "de8e5b20-89a3-4262-8e63-def07d1c9634" (UID: "de8e5b20-89a3-4262-8e63-def07d1c9634"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:38:24.759171 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.759145    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-token\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:24.759171 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.759167    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-podinfo\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:24.759171 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.759176    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-data\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:24.759420 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.759185    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-workload-socket\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:24.759420 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.759192    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de8e5b20-89a3-4262-8e63-def07d1c9634-istio-envoy\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:24.759420 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.759203    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgsfq\" (UniqueName: \"kubernetes.io/projected/de8e5b20-89a3-4262-8e63-def07d1c9634-kube-api-access-zgsfq\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:24.934228 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.934145    2575 generic.go:356] "Generic (PLEG): container finished" podID="de8e5b20-89a3-4262-8e63-def07d1c9634" containerID="c2b45f9310297b5790ddc4eb40560022e09590c25d428f5516d1166937e56e5e" exitCode=0
Jul 15 17:38:24.934228 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.934187    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57" event={"ID":"de8e5b20-89a3-4262-8e63-def07d1c9634","Type":"ContainerDied","Data":"c2b45f9310297b5790ddc4eb40560022e09590c25d428f5516d1166937e56e5e"}
Jul 15 17:38:24.934228 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.934192    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"
Jul 15 17:38:24.934503 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.934244    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57" event={"ID":"de8e5b20-89a3-4262-8e63-def07d1c9634","Type":"ContainerDied","Data":"d7e32eafa1e98646c7979610d342461deb0f135d2dc23f49fb5b634bc6bf1500"}
Jul 15 17:38:24.934503 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.934271    2575 scope.go:117] "RemoveContainer" containerID="c2b45f9310297b5790ddc4eb40560022e09590c25d428f5516d1166937e56e5e"
Jul 15 17:38:24.943427 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.943409    2575 scope.go:117] "RemoveContainer" containerID="f0244fb2c6fc77e432c735e933967092fe11104a2e615e8ceb7e53f5b61a037f"
Jul 15 17:38:24.952867 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.952847    2575 scope.go:117] "RemoveContainer" containerID="c2b45f9310297b5790ddc4eb40560022e09590c25d428f5516d1166937e56e5e"
Jul 15 17:38:24.953308 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:38:24.953285    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2b45f9310297b5790ddc4eb40560022e09590c25d428f5516d1166937e56e5e\": container with ID starting with c2b45f9310297b5790ddc4eb40560022e09590c25d428f5516d1166937e56e5e not found: ID does not exist" containerID="c2b45f9310297b5790ddc4eb40560022e09590c25d428f5516d1166937e56e5e"
Jul 15 17:38:24.953419 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.953316    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2b45f9310297b5790ddc4eb40560022e09590c25d428f5516d1166937e56e5e"} err="failed to get container status \"c2b45f9310297b5790ddc4eb40560022e09590c25d428f5516d1166937e56e5e\": rpc error: code = NotFound desc = could not find container \"c2b45f9310297b5790ddc4eb40560022e09590c25d428f5516d1166937e56e5e\": container with ID starting with c2b45f9310297b5790ddc4eb40560022e09590c25d428f5516d1166937e56e5e not found: ID does not exist"
Jul 15 17:38:24.953419 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.953334    2575 scope.go:117] "RemoveContainer" containerID="f0244fb2c6fc77e432c735e933967092fe11104a2e615e8ceb7e53f5b61a037f"
Jul 15 17:38:24.953633 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:38:24.953616    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0244fb2c6fc77e432c735e933967092fe11104a2e615e8ceb7e53f5b61a037f\": container with ID starting with f0244fb2c6fc77e432c735e933967092fe11104a2e615e8ceb7e53f5b61a037f not found: ID does not exist" containerID="f0244fb2c6fc77e432c735e933967092fe11104a2e615e8ceb7e53f5b61a037f"
Jul 15 17:38:24.953712 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.953637    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0244fb2c6fc77e432c735e933967092fe11104a2e615e8ceb7e53f5b61a037f"} err="failed to get container status \"f0244fb2c6fc77e432c735e933967092fe11104a2e615e8ceb7e53f5b61a037f\": rpc error: code = NotFound desc = could not find container \"f0244fb2c6fc77e432c735e933967092fe11104a2e615e8ceb7e53f5b61a037f\": container with ID starting with f0244fb2c6fc77e432c735e933967092fe11104a2e615e8ceb7e53f5b61a037f not found: ID does not exist"
Jul 15 17:38:24.968330 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.968309    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"]
Jul 15 17:38:24.979023 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:24.979005    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-b525cf-openshift-default-786f85db6c-nwd57"]
Jul 15 17:38:26.526122 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:26.526091    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de8e5b20-89a3-4262-8e63-def07d1c9634" path="/var/lib/kubelet/pods/de8e5b20-89a3-4262-8e63-def07d1c9634/volumes"
Jul 15 17:38:30.510308 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:30.510269    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d"]
Jul 15 17:38:30.510722 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:30.510540    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d" podUID="87abafec-8da3-40bb-a396-fc0750c6445f" containerName="maas-api" containerID="cri-o://5c7a3e2b61a35e2390d01512b135aaf35a54d723ebeb8f89fca51cec470f1621" gracePeriod=30
Jul 15 17:38:30.757479 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:30.757454    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d"
Jul 15 17:38:30.913719 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:30.913692    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5nw9x\" (UniqueName: \"kubernetes.io/projected/87abafec-8da3-40bb-a396-fc0750c6445f-kube-api-access-5nw9x\") pod \"87abafec-8da3-40bb-a396-fc0750c6445f\" (UID: \"87abafec-8da3-40bb-a396-fc0750c6445f\") "
Jul 15 17:38:30.913869 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:30.913767    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87abafec-8da3-40bb-a396-fc0750c6445f-maas-api-tls\") pod \"87abafec-8da3-40bb-a396-fc0750c6445f\" (UID: \"87abafec-8da3-40bb-a396-fc0750c6445f\") "
Jul 15 17:38:30.913869 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:30.913799    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87abafec-8da3-40bb-a396-fc0750c6445f-openshift-service-ca\") pod \"87abafec-8da3-40bb-a396-fc0750c6445f\" (UID: \"87abafec-8da3-40bb-a396-fc0750c6445f\") "
Jul 15 17:38:30.914169 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:30.914132    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87abafec-8da3-40bb-a396-fc0750c6445f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "87abafec-8da3-40bb-a396-fc0750c6445f" (UID: "87abafec-8da3-40bb-a396-fc0750c6445f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:38:30.915897 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:30.915873    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87abafec-8da3-40bb-a396-fc0750c6445f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "87abafec-8da3-40bb-a396-fc0750c6445f" (UID: "87abafec-8da3-40bb-a396-fc0750c6445f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:38:30.915981 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:30.915909    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87abafec-8da3-40bb-a396-fc0750c6445f-kube-api-access-5nw9x" (OuterVolumeSpecName: "kube-api-access-5nw9x") pod "87abafec-8da3-40bb-a396-fc0750c6445f" (UID: "87abafec-8da3-40bb-a396-fc0750c6445f"). InnerVolumeSpecName "kube-api-access-5nw9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:38:30.965307 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:30.965275    2575 generic.go:356] "Generic (PLEG): container finished" podID="87abafec-8da3-40bb-a396-fc0750c6445f" containerID="5c7a3e2b61a35e2390d01512b135aaf35a54d723ebeb8f89fca51cec470f1621" exitCode=0
Jul 15 17:38:30.965449 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:30.965313    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d"
Jul 15 17:38:30.965449 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:30.965314    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d" event={"ID":"87abafec-8da3-40bb-a396-fc0750c6445f","Type":"ContainerDied","Data":"5c7a3e2b61a35e2390d01512b135aaf35a54d723ebeb8f89fca51cec470f1621"}
Jul 15 17:38:30.965449 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:30.965440    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d" event={"ID":"87abafec-8da3-40bb-a396-fc0750c6445f","Type":"ContainerDied","Data":"ff2f6abf7256949d019c78a44eea930aab0865b1c0b823d4e86f13de976a8587"}
Jul 15 17:38:30.965579 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:30.965468    2575 scope.go:117] "RemoveContainer" containerID="5c7a3e2b61a35e2390d01512b135aaf35a54d723ebeb8f89fca51cec470f1621"
Jul 15 17:38:30.974667 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:30.974647    2575 scope.go:117] "RemoveContainer" containerID="5c7a3e2b61a35e2390d01512b135aaf35a54d723ebeb8f89fca51cec470f1621"
Jul 15 17:38:30.974903 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:38:30.974886    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c7a3e2b61a35e2390d01512b135aaf35a54d723ebeb8f89fca51cec470f1621\": container with ID starting with 5c7a3e2b61a35e2390d01512b135aaf35a54d723ebeb8f89fca51cec470f1621 not found: ID does not exist" containerID="5c7a3e2b61a35e2390d01512b135aaf35a54d723ebeb8f89fca51cec470f1621"
Jul 15 17:38:30.974956 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:30.974910    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c7a3e2b61a35e2390d01512b135aaf35a54d723ebeb8f89fca51cec470f1621"} err="failed to get container status \"5c7a3e2b61a35e2390d01512b135aaf35a54d723ebeb8f89fca51cec470f1621\": rpc error: code = NotFound desc = could not find container \"5c7a3e2b61a35e2390d01512b135aaf35a54d723ebeb8f89fca51cec470f1621\": container with ID starting with 5c7a3e2b61a35e2390d01512b135aaf35a54d723ebeb8f89fca51cec470f1621 not found: ID does not exist"
Jul 15 17:38:31.014919 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:31.014897    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87abafec-8da3-40bb-a396-fc0750c6445f-openshift-service-ca\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:31.014919 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:31.014917    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nw9x\" (UniqueName: \"kubernetes.io/projected/87abafec-8da3-40bb-a396-fc0750c6445f-kube-api-access-5nw9x\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:31.015091 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:31.014928    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87abafec-8da3-40bb-a396-fc0750c6445f-maas-api-tls\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:38:31.025861 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:31.025831    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d"]
Jul 15 17:38:31.032185 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:31.032165    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ed384a-5bd5d5c5d4-vjc5d"]
Jul 15 17:38:32.525746 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:32.525714    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87abafec-8da3-40bb-a396-fc0750c6445f" path="/var/lib/kubelet/pods/87abafec-8da3-40bb-a396-fc0750c6445f/volumes"
Jul 15 17:38:46.736143 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.736111    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"]
Jul 15 17:38:46.736614 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.736463    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de8e5b20-89a3-4262-8e63-def07d1c9634" containerName="istio-proxy"
Jul 15 17:38:46.736614 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.736474    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="de8e5b20-89a3-4262-8e63-def07d1c9634" containerName="istio-proxy"
Jul 15 17:38:46.736614 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.736486    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c9c9943-7381-4752-9f3c-8984f96ae2ca" containerName="main"
Jul 15 17:38:46.736614 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.736491    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c9c9943-7381-4752-9f3c-8984f96ae2ca" containerName="main"
Jul 15 17:38:46.736614 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.736500    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dc4fbcb-8753-44fe-86eb-fd99d8eac40e" containerName="main"
Jul 15 17:38:46.736614 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.736507    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dc4fbcb-8753-44fe-86eb-fd99d8eac40e" containerName="main"
Jul 15 17:38:46.736614 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.736533    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87abafec-8da3-40bb-a396-fc0750c6445f" containerName="maas-api"
Jul 15 17:38:46.736614 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.736538    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="87abafec-8da3-40bb-a396-fc0750c6445f" containerName="maas-api"
Jul 15 17:38:46.736884 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.736648    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="de8e5b20-89a3-4262-8e63-def07d1c9634" containerName="istio-proxy"
Jul 15 17:38:46.736884 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.736659    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="87abafec-8da3-40bb-a396-fc0750c6445f" containerName="maas-api"
Jul 15 17:38:46.736884 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.736669    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c9c9943-7381-4752-9f3c-8984f96ae2ca" containerName="main"
Jul 15 17:38:46.736884 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.736683    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dc4fbcb-8753-44fe-86eb-fd99d8eac40e" containerName="main"
Jul 15 17:38:46.741366 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.741347    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.747499 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.747476    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-d83283-openshift-default-dockercfg-bfq8l\""
Jul 15 17:38:46.750711 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.750688    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a072c38c-62cd-470f-9d3e-07143e247aa6-istiod-ca-cert\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.750827 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.750753    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-workload-certs\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.750827 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.750782    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-workload-socket\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.750827 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.750819    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc96x\" (UniqueName: \"kubernetes.io/projected/a072c38c-62cd-470f-9d3e-07143e247aa6-kube-api-access-nc96x\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.750962 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.750863    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-token\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.750962 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.750908    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-envoy\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.751049 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.750968    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-data\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.751093 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.751084    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-credential-socket\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.751128 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.751107    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-podinfo\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.807347 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.807320    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"]
Jul 15 17:38:46.851421 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.851388    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-credential-socket\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.851641 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.851424    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-podinfo\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.851641 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.851452    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a072c38c-62cd-470f-9d3e-07143e247aa6-istiod-ca-cert\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.851641 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.851486    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-workload-certs\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.851641 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.851502    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-workload-socket\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.851641 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.851533    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc96x\" (UniqueName: \"kubernetes.io/projected/a072c38c-62cd-470f-9d3e-07143e247aa6-kube-api-access-nc96x\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.851641 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.851585    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-token\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.851641 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.851636    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-envoy\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.852028 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.851699    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-data\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.852028 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.851952    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-credential-socket\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.852028 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.851999    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-workload-certs\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.852182 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.852064    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-workload-socket\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.852234 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.852217    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-data\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.852340 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.852323    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a072c38c-62cd-470f-9d3e-07143e247aa6-istiod-ca-cert\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.853917 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.853900    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-envoy\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.853990 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.853972    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-podinfo\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.866086 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.866064    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-token\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:46.867107 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:46.867089    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc96x\" (UniqueName: \"kubernetes.io/projected/a072c38c-62cd-470f-9d3e-07143e247aa6-kube-api-access-nc96x\") pod \"e2e-api-a-d83283-openshift-default-7c445b67-bhbp7\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") " pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:47.052269 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:47.052188    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:47.199200 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:47.199178    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"]
Jul 15 17:38:47.201904 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:38:47.201876    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda072c38c_62cd_470f_9d3e_07143e247aa6.slice/crio-ffbc66873f23f0650e168c0cf83f42b6d88032cb4b55f8e1ac9f4474aee88ca4 WatchSource:0}: Error finding container ffbc66873f23f0650e168c0cf83f42b6d88032cb4b55f8e1ac9f4474aee88ca4: Status 404 returned error can't find the container with id ffbc66873f23f0650e168c0cf83f42b6d88032cb4b55f8e1ac9f4474aee88ca4
Jul 15 17:38:47.203982 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:47.203953    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:38:47.204046 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:47.204016    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:38:47.204046 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:47.204043    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:38:48.050978 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:48.050943    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7" event={"ID":"a072c38c-62cd-470f-9d3e-07143e247aa6","Type":"ContainerStarted","Data":"01a38de11387a1501f02283104a2e4795b3f0d8be2fbfa10117988374bb188cf"}
Jul 15 17:38:48.051349 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:48.050995    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7" event={"ID":"a072c38c-62cd-470f-9d3e-07143e247aa6","Type":"ContainerStarted","Data":"ffbc66873f23f0650e168c0cf83f42b6d88032cb4b55f8e1ac9f4474aee88ca4"}
Jul 15 17:38:48.053314 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:48.053288    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:48.058066 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:48.058044    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:48.162075 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:48.162030    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7" podStartSLOduration=2.162014026 podStartE2EDuration="2.162014026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:38:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:38:48.094889743 +0000 UTC m=+2544.112302839" watchObservedRunningTime="2026-07-15 17:38:48.162014026 +0000 UTC m=+2544.179427123"
Jul 15 17:38:49.055495 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:49.055465    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:49.056582 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:49.056548    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:38:50.594444 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:50.594412    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"]
Jul 15 17:38:50.594990 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:50.594971    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de8e5b20-89a3-4262-8e63-def07d1c9634" containerName="istio-proxy"
Jul 15 17:38:50.595079 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:50.594992    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="de8e5b20-89a3-4262-8e63-def07d1c9634" containerName="istio-proxy"
Jul 15 17:38:50.595133 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:50.595105    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="de8e5b20-89a3-4262-8e63-def07d1c9634" containerName="istio-proxy"
Jul 15 17:38:50.604975 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:50.604948    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"
Jul 15 17:38:50.607887 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:38:50.607852    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p\" is forbidden: User \"system:node:ip-10-0-142-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-142-51.ec2.internal' and this object" podUID="03249797-3ba5-403c-8c99-d29258acc2ac" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"
Jul 15 17:38:50.607887 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:38:50.607871    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-api-e2e-api-a-d83283-serving-cert\" is forbidden: User \"system:node:ip-10-0-142-51.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-142-51.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-d83283-serving-cert\"" type="*v1.Secret"
Jul 15 17:38:50.625459 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:50.625430    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"]
Jul 15 17:38:50.685964 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:50.685934    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03249797-3ba5-403c-8c99-d29258acc2ac-openshift-service-ca\") pod \"maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p\" (UID: \"03249797-3ba5-403c-8c99-d29258acc2ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"
Jul 15 17:38:50.686091 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:50.685974    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9w86\" (UniqueName: \"kubernetes.io/projected/03249797-3ba5-403c-8c99-d29258acc2ac-kube-api-access-d9w86\") pod \"maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p\" (UID: \"03249797-3ba5-403c-8c99-d29258acc2ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"
Jul 15 17:38:50.686091 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:50.686019    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03249797-3ba5-403c-8c99-d29258acc2ac-maas-api-tls\") pod \"maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p\" (UID: \"03249797-3ba5-403c-8c99-d29258acc2ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"
Jul 15 17:38:50.786983 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:50.786952    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03249797-3ba5-403c-8c99-d29258acc2ac-openshift-service-ca\") pod \"maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p\" (UID: \"03249797-3ba5-403c-8c99-d29258acc2ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"
Jul 15 17:38:50.787143 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:50.786993    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9w86\" (UniqueName: \"kubernetes.io/projected/03249797-3ba5-403c-8c99-d29258acc2ac-kube-api-access-d9w86\") pod \"maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p\" (UID: \"03249797-3ba5-403c-8c99-d29258acc2ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"
Jul 15 17:38:50.787143 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:50.787030    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03249797-3ba5-403c-8c99-d29258acc2ac-maas-api-tls\") pod \"maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p\" (UID: \"03249797-3ba5-403c-8c99-d29258acc2ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"
Jul 15 17:38:50.787612 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:50.787552    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03249797-3ba5-403c-8c99-d29258acc2ac-openshift-service-ca\") pod \"maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p\" (UID: \"03249797-3ba5-403c-8c99-d29258acc2ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"
Jul 15 17:38:50.818657 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:50.818634    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9w86\" (UniqueName: \"kubernetes.io/projected/03249797-3ba5-403c-8c99-d29258acc2ac-kube-api-access-d9w86\") pod \"maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p\" (UID: \"03249797-3ba5-403c-8c99-d29258acc2ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"
Jul 15 17:38:51.787437 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:38:51.787406    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-serving-cert: failed to sync secret cache: timed out waiting for the condition
Jul 15 17:38:51.787837 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:38:51.787493    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03249797-3ba5-403c-8c99-d29258acc2ac-maas-api-tls podName:03249797-3ba5-403c-8c99-d29258acc2ac nodeName:}" failed. No retries permitted until 2026-07-15 17:38:52.287473307 +0000 UTC m=+2548.304886386 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/03249797-3ba5-403c-8c99-d29258acc2ac-maas-api-tls") pod "maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p" (UID: "03249797-3ba5-403c-8c99-d29258acc2ac") : failed to sync secret cache: timed out waiting for the condition
Jul 15 17:38:51.878237 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:51.878167    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-d83283-serving-cert\""
Jul 15 17:38:52.300379 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:52.300299    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03249797-3ba5-403c-8c99-d29258acc2ac-maas-api-tls\") pod \"maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p\" (UID: \"03249797-3ba5-403c-8c99-d29258acc2ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"
Jul 15 17:38:52.302818 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:52.302789    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03249797-3ba5-403c-8c99-d29258acc2ac-maas-api-tls\") pod \"maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p\" (UID: \"03249797-3ba5-403c-8c99-d29258acc2ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"
Jul 15 17:38:52.418504 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:52.418467    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"
Jul 15 17:38:52.586023 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:52.585998    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"]
Jul 15 17:38:52.586743 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:38:52.586715    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03249797_3ba5_403c_8c99_d29258acc2ac.slice/crio-c4241dece76dd0d0a57e52a5042ee668a84e767beb50662116a3dc7d8880e489 WatchSource:0}: Error finding container c4241dece76dd0d0a57e52a5042ee668a84e767beb50662116a3dc7d8880e489: Status 404 returned error can't find the container with id c4241dece76dd0d0a57e52a5042ee668a84e767beb50662116a3dc7d8880e489
Jul 15 17:38:52.588280 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:52.588263    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 17:38:53.076319 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:53.076286    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p" event={"ID":"03249797-3ba5-403c-8c99-d29258acc2ac","Type":"ContainerStarted","Data":"c4241dece76dd0d0a57e52a5042ee668a84e767beb50662116a3dc7d8880e489"}
Jul 15 17:38:54.082381 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:54.082296    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p" event={"ID":"03249797-3ba5-403c-8c99-d29258acc2ac","Type":"ContainerStarted","Data":"b6ff867f4708be5f903edf006cd7927cc7dd6196d65ab6de274283e4c84ceed0"}
Jul 15 17:38:54.082743 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:54.082417    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"
Jul 15 17:38:54.109805 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:54.109754    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p" podStartSLOduration=2.987124104 podStartE2EDuration="4.109738601s" totalImagesPullingTime="1.122614497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:38:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:38:54.108178577 +0000 UTC m=+2550.125591704" watchObservedRunningTime="2026-07-15 17:38:54.109738601 +0000 UTC m=+2550.127151697"
Jul 15 17:38:56.485394 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.485360    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"]
Jul 15 17:38:56.489013 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.488996    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.492602 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.492579    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-1da9d9-openshift-default-dockercfg-sl564\""
Jul 15 17:38:56.526489 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.526460    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"]
Jul 15 17:38:56.536598 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.536549    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-workload-certs\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.536748 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.536617    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d18c08f7-1996-4d15-928a-925dd24b2dad-istiod-ca-cert\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.536748 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.536644    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx64b\" (UniqueName: \"kubernetes.io/projected/d18c08f7-1996-4d15-928a-925dd24b2dad-kube-api-access-sx64b\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.536748 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.536675    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-envoy\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.536870 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.536801    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-podinfo\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.536870 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.536844    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-workload-socket\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.536947 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.536917    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-data\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.536990 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.536943    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-credential-socket\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.537033 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.536989    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-token\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.638309 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.638275    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-workload-certs\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.638309 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.638315    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d18c08f7-1996-4d15-928a-925dd24b2dad-istiod-ca-cert\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.639198 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.638333    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx64b\" (UniqueName: \"kubernetes.io/projected/d18c08f7-1996-4d15-928a-925dd24b2dad-kube-api-access-sx64b\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.639198 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.638358    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-envoy\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.639198 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.638406    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-podinfo\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.639198 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.638425    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-workload-socket\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.639198 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.638449    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-data\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.639198 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.638537    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-credential-socket\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.639198 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.638636    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-token\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.639198 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.639171    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-workload-certs\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.639198 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.639188    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-credential-socket\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.639198 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.639198    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d18c08f7-1996-4d15-928a-925dd24b2dad-istiod-ca-cert\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.639592 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.639257    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-data\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.639592 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.639286    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-workload-socket\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.641153 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.641114    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-podinfo\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.641294 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.641272    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-envoy\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.654521 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.654497    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx64b\" (UniqueName: \"kubernetes.io/projected/d18c08f7-1996-4d15-928a-925dd24b2dad-kube-api-access-sx64b\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.660165 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.660142    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-token\") pod \"e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") " pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.799785 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.799706    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:56.997514 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:56.997484    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"]
Jul 15 17:38:57.001187 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:38:57.001163    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd18c08f7_1996_4d15_928a_925dd24b2dad.slice/crio-c838f5512e2ad06518be8388755b2435a6b8c1e72666d41799e65b3d8b19bc1a WatchSource:0}: Error finding container c838f5512e2ad06518be8388755b2435a6b8c1e72666d41799e65b3d8b19bc1a: Status 404 returned error can't find the container with id c838f5512e2ad06518be8388755b2435a6b8c1e72666d41799e65b3d8b19bc1a
Jul 15 17:38:57.003455 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:57.003422    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:38:57.003524 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:57.003487    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:38:57.003524 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:57.003517    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:38:57.097929 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:57.097904    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5" event={"ID":"d18c08f7-1996-4d15-928a-925dd24b2dad","Type":"ContainerStarted","Data":"c838f5512e2ad06518be8388755b2435a6b8c1e72666d41799e65b3d8b19bc1a"}
Jul 15 17:38:58.104842 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:58.104806    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5" event={"ID":"d18c08f7-1996-4d15-928a-925dd24b2dad","Type":"ContainerStarted","Data":"dd654189e9a9332f1bb35a78e07e60d2c7855e92036203affc91e6e6dfe27afb"}
Jul 15 17:38:58.127358 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:58.127275    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5" podStartSLOduration=2.127257288 podStartE2EDuration="2.127257288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:38:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:38:58.126428038 +0000 UTC m=+2554.143841134" watchObservedRunningTime="2026-07-15 17:38:58.127257288 +0000 UTC m=+2554.144670388"
Jul 15 17:38:58.800138 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:58.800099    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:58.805040 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:58.805017    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:59.109864 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:59.109776    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:38:59.110700 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:38:59.110679    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:39:00.092832 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:00.092803    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"
Jul 15 17:39:04.056945 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:04.056867    2575 patch_prober.go:28] interesting pod/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 15 17:39:04.057325 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:04.056949    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7" podUID="a072c38c-62cd-470f-9d3e-07143e247aa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 15 17:39:04.136644 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:04.136546    2575 generic.go:356] "Generic (PLEG): container finished" podID="a072c38c-62cd-470f-9d3e-07143e247aa6" containerID="01a38de11387a1501f02283104a2e4795b3f0d8be2fbfa10117988374bb188cf" exitCode=0
Jul 15 17:39:04.136644 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:04.136594    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7" event={"ID":"a072c38c-62cd-470f-9d3e-07143e247aa6","Type":"ContainerDied","Data":"01a38de11387a1501f02283104a2e4795b3f0d8be2fbfa10117988374bb188cf"}
Jul 15 17:39:04.137026 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:04.137012    2575 scope.go:117] "RemoveContainer" containerID="01a38de11387a1501f02283104a2e4795b3f0d8be2fbfa10117988374bb188cf"
Jul 15 17:39:04.138038 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:04.138009    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:39:04.138140 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:04.138090    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:39:04.138207 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:04.138136    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:39:05.145123 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:05.145086    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7" event={"ID":"a072c38c-62cd-470f-9d3e-07143e247aa6","Type":"ContainerStarted","Data":"e455968209899a20ad9cf139783116466136078184cbab1919525d162d1ccba1"}
Jul 15 17:39:06.053782 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:06.053336    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:39:06.053782 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:06.053455    2575 patch_prober.go:28] interesting pod/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 15 17:39:06.053782 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:06.053500    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7" podUID="a072c38c-62cd-470f-9d3e-07143e247aa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 15 17:39:07.053405 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:07.053360    2575 patch_prober.go:28] interesting pod/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 15 17:39:07.053943 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:07.053434    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7" podUID="a072c38c-62cd-470f-9d3e-07143e247aa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 15 17:39:08.052952 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:08.052912    2575 patch_prober.go:28] interesting pod/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 15 17:39:08.053663 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:08.053235    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7" podUID="a072c38c-62cd-470f-9d3e-07143e247aa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 15 17:39:09.053296 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:09.053255    2575 patch_prober.go:28] interesting pod/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 15 17:39:09.053641 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:09.053609    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7" podUID="a072c38c-62cd-470f-9d3e-07143e247aa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 15 17:39:10.066781 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:10.066746    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:39:10.067268 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:10.067130    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:39:10.067873 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:10.067854    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:39:15.150739 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:15.150700    2575 patch_prober.go:28] interesting pod/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 15 17:39:15.151206 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:15.150777    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5" podUID="d18c08f7-1996-4d15-928a-925dd24b2dad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 15 17:39:17.214504 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:17.214411    2575 generic.go:356] "Generic (PLEG): container finished" podID="d18c08f7-1996-4d15-928a-925dd24b2dad" containerID="dd654189e9a9332f1bb35a78e07e60d2c7855e92036203affc91e6e6dfe27afb" exitCode=0
Jul 15 17:39:17.214504 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:17.214485    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5" event={"ID":"d18c08f7-1996-4d15-928a-925dd24b2dad","Type":"ContainerDied","Data":"dd654189e9a9332f1bb35a78e07e60d2c7855e92036203affc91e6e6dfe27afb"}
Jul 15 17:39:17.214974 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:17.214959    2575 scope.go:117] "RemoveContainer" containerID="dd654189e9a9332f1bb35a78e07e60d2c7855e92036203affc91e6e6dfe27afb"
Jul 15 17:39:17.216020 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:17.215979    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:39:17.216098 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:17.216079    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:39:17.216138 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:17.216119    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:39:18.221819 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:18.221789    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5" event={"ID":"d18c08f7-1996-4d15-928a-925dd24b2dad","Type":"ContainerStarted","Data":"e8d38b1e5f1adcdaf8d0cd8c10db4b5adb77b68febda894f35f79175530db631"}
Jul 15 17:39:18.800438 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:18.800406    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:39:18.800642 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:18.800599    2575 patch_prober.go:28] interesting pod/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 15 17:39:18.800739 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:18.800657    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5" podUID="d18c08f7-1996-4d15-928a-925dd24b2dad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 15 17:39:19.800618 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:19.800588    2575 patch_prober.go:28] interesting pod/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 15 17:39:19.800992 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:19.800647    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5" podUID="d18c08f7-1996-4d15-928a-925dd24b2dad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 15 17:39:20.800276 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:20.800240    2575 patch_prober.go:28] interesting pod/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 15 17:39:20.800459 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:20.800286    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5" podUID="d18c08f7-1996-4d15-928a-925dd24b2dad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 15 17:39:21.800981 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:21.800942    2575 patch_prober.go:28] interesting pod/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 15 17:39:21.801366 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:21.801016    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5" podUID="d18c08f7-1996-4d15-928a-925dd24b2dad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 15 17:39:22.803532 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:22.803499    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:39:22.803926 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:22.803875    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:39:22.804546 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:22.804525    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:39:24.441041 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:24.441006    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"]
Jul 15 17:39:24.441473 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:24.441247    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p" podUID="03249797-3ba5-403c-8c99-d29258acc2ac" containerName="maas-api" containerID="cri-o://b6ff867f4708be5f903edf006cd7927cc7dd6196d65ab6de274283e4c84ceed0" gracePeriod=30
Jul 15 17:39:24.701625 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:24.701527    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"
Jul 15 17:39:24.846629 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:24.846593    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03249797-3ba5-403c-8c99-d29258acc2ac-maas-api-tls\") pod \"03249797-3ba5-403c-8c99-d29258acc2ac\" (UID: \"03249797-3ba5-403c-8c99-d29258acc2ac\") "
Jul 15 17:39:24.846803 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:24.846724    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03249797-3ba5-403c-8c99-d29258acc2ac-openshift-service-ca\") pod \"03249797-3ba5-403c-8c99-d29258acc2ac\" (UID: \"03249797-3ba5-403c-8c99-d29258acc2ac\") "
Jul 15 17:39:24.846803 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:24.846751    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9w86\" (UniqueName: \"kubernetes.io/projected/03249797-3ba5-403c-8c99-d29258acc2ac-kube-api-access-d9w86\") pod \"03249797-3ba5-403c-8c99-d29258acc2ac\" (UID: \"03249797-3ba5-403c-8c99-d29258acc2ac\") "
Jul 15 17:39:24.847109 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:24.847081    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03249797-3ba5-403c-8c99-d29258acc2ac-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "03249797-3ba5-403c-8c99-d29258acc2ac" (UID: "03249797-3ba5-403c-8c99-d29258acc2ac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:39:24.848551 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:24.848522    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03249797-3ba5-403c-8c99-d29258acc2ac-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "03249797-3ba5-403c-8c99-d29258acc2ac" (UID: "03249797-3ba5-403c-8c99-d29258acc2ac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:39:24.849065 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:24.849039    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03249797-3ba5-403c-8c99-d29258acc2ac-kube-api-access-d9w86" (OuterVolumeSpecName: "kube-api-access-d9w86") pod "03249797-3ba5-403c-8c99-d29258acc2ac" (UID: "03249797-3ba5-403c-8c99-d29258acc2ac"). InnerVolumeSpecName "kube-api-access-d9w86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:39:24.948454 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:24.948421    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03249797-3ba5-403c-8c99-d29258acc2ac-openshift-service-ca\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:39:24.948454 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:24.948445    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9w86\" (UniqueName: \"kubernetes.io/projected/03249797-3ba5-403c-8c99-d29258acc2ac-kube-api-access-d9w86\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:39:24.948454 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:24.948456    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03249797-3ba5-403c-8c99-d29258acc2ac-maas-api-tls\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:39:25.256699 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:25.256668    2575 generic.go:356] "Generic (PLEG): container finished" podID="03249797-3ba5-403c-8c99-d29258acc2ac" containerID="b6ff867f4708be5f903edf006cd7927cc7dd6196d65ab6de274283e4c84ceed0" exitCode=0
Jul 15 17:39:25.256861 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:25.256709    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"
Jul 15 17:39:25.256861 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:25.256725    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p" event={"ID":"03249797-3ba5-403c-8c99-d29258acc2ac","Type":"ContainerDied","Data":"b6ff867f4708be5f903edf006cd7927cc7dd6196d65ab6de274283e4c84ceed0"}
Jul 15 17:39:25.256861 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:25.256766    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p" event={"ID":"03249797-3ba5-403c-8c99-d29258acc2ac","Type":"ContainerDied","Data":"c4241dece76dd0d0a57e52a5042ee668a84e767beb50662116a3dc7d8880e489"}
Jul 15 17:39:25.256861 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:25.256786    2575 scope.go:117] "RemoveContainer" containerID="b6ff867f4708be5f903edf006cd7927cc7dd6196d65ab6de274283e4c84ceed0"
Jul 15 17:39:25.266239 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:25.266224    2575 scope.go:117] "RemoveContainer" containerID="b6ff867f4708be5f903edf006cd7927cc7dd6196d65ab6de274283e4c84ceed0"
Jul 15 17:39:25.266482 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:39:25.266465    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6ff867f4708be5f903edf006cd7927cc7dd6196d65ab6de274283e4c84ceed0\": container with ID starting with b6ff867f4708be5f903edf006cd7927cc7dd6196d65ab6de274283e4c84ceed0 not found: ID does not exist" containerID="b6ff867f4708be5f903edf006cd7927cc7dd6196d65ab6de274283e4c84ceed0"
Jul 15 17:39:25.266524 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:25.266491    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6ff867f4708be5f903edf006cd7927cc7dd6196d65ab6de274283e4c84ceed0"} err="failed to get container status \"b6ff867f4708be5f903edf006cd7927cc7dd6196d65ab6de274283e4c84ceed0\": rpc error: code = NotFound desc = could not find container \"b6ff867f4708be5f903edf006cd7927cc7dd6196d65ab6de274283e4c84ceed0\": container with ID starting with b6ff867f4708be5f903edf006cd7927cc7dd6196d65ab6de274283e4c84ceed0 not found: ID does not exist"
Jul 15 17:39:25.278932 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:25.278906    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"]
Jul 15 17:39:25.280959 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:25.280938    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d83283-7f6f6d775f-8cz9p"]
Jul 15 17:39:26.526245 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:26.526212    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03249797-3ba5-403c-8c99-d29258acc2ac" path="/var/lib/kubelet/pods/03249797-3ba5-403c-8c99-d29258acc2ac/volumes"
Jul 15 17:39:39.452246 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:39.452211    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"]
Jul 15 17:39:39.453084 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:39.453046    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7" podUID="a072c38c-62cd-470f-9d3e-07143e247aa6" containerName="istio-proxy" containerID="cri-o://e455968209899a20ad9cf139783116466136078184cbab1919525d162d1ccba1" gracePeriod=30
Jul 15 17:39:40.069335 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:40.069294    2575 patch_prober.go:28] interesting pod/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 17:39:40.069513 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:40.069356    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7" podUID="a072c38c-62cd-470f-9d3e-07143e247aa6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 17:39:44.731894 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.731820    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:39:44.812426 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.812393    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-workload-socket\") pod \"a072c38c-62cd-470f-9d3e-07143e247aa6\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") "
Jul 15 17:39:44.812426 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.812430    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-token\") pod \"a072c38c-62cd-470f-9d3e-07143e247aa6\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") "
Jul 15 17:39:44.812674 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.812446    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-envoy\") pod \"a072c38c-62cd-470f-9d3e-07143e247aa6\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") "
Jul 15 17:39:44.812674 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.812500    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nc96x\" (UniqueName: \"kubernetes.io/projected/a072c38c-62cd-470f-9d3e-07143e247aa6-kube-api-access-nc96x\") pod \"a072c38c-62cd-470f-9d3e-07143e247aa6\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") "
Jul 15 17:39:44.812674 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.812517    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-podinfo\") pod \"a072c38c-62cd-470f-9d3e-07143e247aa6\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") "
Jul 15 17:39:44.812674 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.812541    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-workload-certs\") pod \"a072c38c-62cd-470f-9d3e-07143e247aa6\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") "
Jul 15 17:39:44.812674 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.812587    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-data\") pod \"a072c38c-62cd-470f-9d3e-07143e247aa6\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") "
Jul 15 17:39:44.812674 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.812647    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-credential-socket\") pod \"a072c38c-62cd-470f-9d3e-07143e247aa6\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") "
Jul 15 17:39:44.812674 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.812679    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a072c38c-62cd-470f-9d3e-07143e247aa6-istiod-ca-cert\") pod \"a072c38c-62cd-470f-9d3e-07143e247aa6\" (UID: \"a072c38c-62cd-470f-9d3e-07143e247aa6\") "
Jul 15 17:39:44.813061 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.812704    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a072c38c-62cd-470f-9d3e-07143e247aa6" (UID: "a072c38c-62cd-470f-9d3e-07143e247aa6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:39:44.813061 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.812934    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a072c38c-62cd-470f-9d3e-07143e247aa6" (UID: "a072c38c-62cd-470f-9d3e-07143e247aa6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:39:44.813061 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.812958    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a072c38c-62cd-470f-9d3e-07143e247aa6" (UID: "a072c38c-62cd-470f-9d3e-07143e247aa6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:39:44.813061 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.812976    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-workload-socket\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:39:44.813269 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.813225    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a072c38c-62cd-470f-9d3e-07143e247aa6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a072c38c-62cd-470f-9d3e-07143e247aa6" (UID: "a072c38c-62cd-470f-9d3e-07143e247aa6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:39:44.813525 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.813503    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-data" (OuterVolumeSpecName: "istio-data") pod "a072c38c-62cd-470f-9d3e-07143e247aa6" (UID: "a072c38c-62cd-470f-9d3e-07143e247aa6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:39:44.814873 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.814843    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a072c38c-62cd-470f-9d3e-07143e247aa6" (UID: "a072c38c-62cd-470f-9d3e-07143e247aa6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 17:39:44.814978 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.814885    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a072c38c-62cd-470f-9d3e-07143e247aa6-kube-api-access-nc96x" (OuterVolumeSpecName: "kube-api-access-nc96x") pod "a072c38c-62cd-470f-9d3e-07143e247aa6" (UID: "a072c38c-62cd-470f-9d3e-07143e247aa6"). InnerVolumeSpecName "kube-api-access-nc96x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:39:44.815060 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.815041    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a072c38c-62cd-470f-9d3e-07143e247aa6" (UID: "a072c38c-62cd-470f-9d3e-07143e247aa6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:39:44.815099 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.815087    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-token" (OuterVolumeSpecName: "istio-token") pod "a072c38c-62cd-470f-9d3e-07143e247aa6" (UID: "a072c38c-62cd-470f-9d3e-07143e247aa6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:39:44.913980 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.913949    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-token\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:39:44.913980 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.913980    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-envoy\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:39:44.914170 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.913995    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nc96x\" (UniqueName: \"kubernetes.io/projected/a072c38c-62cd-470f-9d3e-07143e247aa6-kube-api-access-nc96x\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:39:44.914170 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.914011    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-podinfo\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:39:44.914170 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.914024    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-workload-certs\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:39:44.914170 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.914036    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-istio-data\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:39:44.914170 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.914048    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a072c38c-62cd-470f-9d3e-07143e247aa6-credential-socket\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:39:44.914170 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:44.914060    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a072c38c-62cd-470f-9d3e-07143e247aa6-istiod-ca-cert\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:39:45.358443 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:45.358406    2575 generic.go:356] "Generic (PLEG): container finished" podID="a072c38c-62cd-470f-9d3e-07143e247aa6" containerID="e455968209899a20ad9cf139783116466136078184cbab1919525d162d1ccba1" exitCode=0
Jul 15 17:39:45.358660 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:45.358474    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"
Jul 15 17:39:45.358660 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:45.358496    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7" event={"ID":"a072c38c-62cd-470f-9d3e-07143e247aa6","Type":"ContainerDied","Data":"e455968209899a20ad9cf139783116466136078184cbab1919525d162d1ccba1"}
Jul 15 17:39:45.358660 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:45.358593    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7" event={"ID":"a072c38c-62cd-470f-9d3e-07143e247aa6","Type":"ContainerDied","Data":"ffbc66873f23f0650e168c0cf83f42b6d88032cb4b55f8e1ac9f4474aee88ca4"}
Jul 15 17:39:45.358660 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:45.358618    2575 scope.go:117] "RemoveContainer" containerID="e455968209899a20ad9cf139783116466136078184cbab1919525d162d1ccba1"
Jul 15 17:39:45.368395 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:45.368379    2575 scope.go:117] "RemoveContainer" containerID="01a38de11387a1501f02283104a2e4795b3f0d8be2fbfa10117988374bb188cf"
Jul 15 17:39:45.376285 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:45.376267    2575 scope.go:117] "RemoveContainer" containerID="e455968209899a20ad9cf139783116466136078184cbab1919525d162d1ccba1"
Jul 15 17:39:45.376523 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:39:45.376502    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e455968209899a20ad9cf139783116466136078184cbab1919525d162d1ccba1\": container with ID starting with e455968209899a20ad9cf139783116466136078184cbab1919525d162d1ccba1 not found: ID does not exist" containerID="e455968209899a20ad9cf139783116466136078184cbab1919525d162d1ccba1"
Jul 15 17:39:45.376583 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:45.376532    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e455968209899a20ad9cf139783116466136078184cbab1919525d162d1ccba1"} err="failed to get container status \"e455968209899a20ad9cf139783116466136078184cbab1919525d162d1ccba1\": rpc error: code = NotFound desc = could not find container \"e455968209899a20ad9cf139783116466136078184cbab1919525d162d1ccba1\": container with ID starting with e455968209899a20ad9cf139783116466136078184cbab1919525d162d1ccba1 not found: ID does not exist"
Jul 15 17:39:45.376583 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:45.376548    2575 scope.go:117] "RemoveContainer" containerID="01a38de11387a1501f02283104a2e4795b3f0d8be2fbfa10117988374bb188cf"
Jul 15 17:39:45.376808 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:39:45.376792    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01a38de11387a1501f02283104a2e4795b3f0d8be2fbfa10117988374bb188cf\": container with ID starting with 01a38de11387a1501f02283104a2e4795b3f0d8be2fbfa10117988374bb188cf not found: ID does not exist" containerID="01a38de11387a1501f02283104a2e4795b3f0d8be2fbfa10117988374bb188cf"
Jul 15 17:39:45.376851 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:45.376814    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01a38de11387a1501f02283104a2e4795b3f0d8be2fbfa10117988374bb188cf"} err="failed to get container status \"01a38de11387a1501f02283104a2e4795b3f0d8be2fbfa10117988374bb188cf\": rpc error: code = NotFound desc = could not find container \"01a38de11387a1501f02283104a2e4795b3f0d8be2fbfa10117988374bb188cf\": container with ID starting with 01a38de11387a1501f02283104a2e4795b3f0d8be2fbfa10117988374bb188cf not found: ID does not exist"
Jul 15 17:39:45.383596 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:45.383575    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"]
Jul 15 17:39:45.389494 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:45.389472    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-d83283-openshift-default-7c445b67-bhbp7"]
Jul 15 17:39:46.525887 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:39:46.525805    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a072c38c-62cd-470f-9d3e-07143e247aa6" path="/var/lib/kubelet/pods/a072c38c-62cd-470f-9d3e-07143e247aa6/volumes"
Jul 15 17:40:05.557268 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:05.557230    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"]
Jul 15 17:40:05.557808 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:05.557489    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5" podUID="d18c08f7-1996-4d15-928a-925dd24b2dad" containerName="istio-proxy" containerID="cri-o://e8d38b1e5f1adcdaf8d0cd8c10db4b5adb77b68febda894f35f79175530db631" gracePeriod=30
Jul 15 17:40:07.806529 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:07.806491    2575 patch_prober.go:28] interesting pod/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 17:40:07.806949 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:07.806612    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5" podUID="d18c08f7-1996-4d15-928a-925dd24b2dad" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 17:40:09.418809 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:09.418777    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l"]
Jul 15 17:40:09.419180 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:09.419167    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a072c38c-62cd-470f-9d3e-07143e247aa6" containerName="istio-proxy"
Jul 15 17:40:09.419224 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:09.419181    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a072c38c-62cd-470f-9d3e-07143e247aa6" containerName="istio-proxy"
Jul 15 17:40:09.419224 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:09.419193    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03249797-3ba5-403c-8c99-d29258acc2ac" containerName="maas-api"
Jul 15 17:40:09.419224 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:09.419199    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="03249797-3ba5-403c-8c99-d29258acc2ac" containerName="maas-api"
Jul 15 17:40:09.419224 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:09.419220    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a072c38c-62cd-470f-9d3e-07143e247aa6" containerName="istio-proxy"
Jul 15 17:40:09.419352 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:09.419227    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a072c38c-62cd-470f-9d3e-07143e247aa6" containerName="istio-proxy"
Jul 15 17:40:09.419352 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:09.419288    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a072c38c-62cd-470f-9d3e-07143e247aa6" containerName="istio-proxy"
Jul 15 17:40:09.419352 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:09.419295    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="03249797-3ba5-403c-8c99-d29258acc2ac" containerName="maas-api"
Jul 15 17:40:09.419352 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:09.419301    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a072c38c-62cd-470f-9d3e-07143e247aa6" containerName="istio-proxy"
Jul 15 17:40:09.421839 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:09.421818    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l"
Jul 15 17:40:09.424339 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:09.424317    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-3d6c37-serving-cert\""
Jul 15 17:40:09.432197 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:09.432172    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l"]
Jul 15 17:40:09.521310 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:09.521278    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b495ff27-5dff-43f5-b06c-f917cc7f75c7-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l\" (UID: \"b495ff27-5dff-43f5-b06c-f917cc7f75c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l"
Jul 15 17:40:09.521508 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:09.521323    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9bkw\" (UniqueName: \"kubernetes.io/projected/b495ff27-5dff-43f5-b06c-f917cc7f75c7-kube-api-access-p9bkw\") pod \"maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l\" (UID: \"b495ff27-5dff-43f5-b06c-f917cc7f75c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l"
Jul 15 17:40:09.521508 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:09.521378    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b495ff27-5dff-43f5-b06c-f917cc7f75c7-maas-api-tls\") pod \"maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l\" (UID: \"b495ff27-5dff-43f5-b06c-f917cc7f75c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l"
Jul 15 17:40:09.622021 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:09.621983    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b495ff27-5dff-43f5-b06c-f917cc7f75c7-maas-api-tls\") pod \"maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l\" (UID: \"b495ff27-5dff-43f5-b06c-f917cc7f75c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l"
Jul 15 17:40:09.622222 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:40:09.622121    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-serving-cert: secret "maas-api-e2e-shared-a-3d6c37-serving-cert" not found
Jul 15 17:40:09.622222 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:09.622139    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b495ff27-5dff-43f5-b06c-f917cc7f75c7-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l\" (UID: \"b495ff27-5dff-43f5-b06c-f917cc7f75c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l"
Jul 15 17:40:09.622222 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:40:09.622188    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b495ff27-5dff-43f5-b06c-f917cc7f75c7-maas-api-tls podName:b495ff27-5dff-43f5-b06c-f917cc7f75c7 nodeName:}" failed. No retries permitted until 2026-07-15 17:40:10.122167368 +0000 UTC m=+2626.139580458 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b495ff27-5dff-43f5-b06c-f917cc7f75c7-maas-api-tls") pod "maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l" (UID: "b495ff27-5dff-43f5-b06c-f917cc7f75c7") : secret "maas-api-e2e-shared-a-3d6c37-serving-cert" not found
Jul 15 17:40:09.622402 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:09.622242    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9bkw\" (UniqueName: \"kubernetes.io/projected/b495ff27-5dff-43f5-b06c-f917cc7f75c7-kube-api-access-p9bkw\") pod \"maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l\" (UID: \"b495ff27-5dff-43f5-b06c-f917cc7f75c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l"
Jul 15 17:40:09.622765 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:09.622746    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b495ff27-5dff-43f5-b06c-f917cc7f75c7-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l\" (UID: \"b495ff27-5dff-43f5-b06c-f917cc7f75c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l"
Jul 15 17:40:09.633331 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:09.633303    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9bkw\" (UniqueName: \"kubernetes.io/projected/b495ff27-5dff-43f5-b06c-f917cc7f75c7-kube-api-access-p9bkw\") pod \"maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l\" (UID: \"b495ff27-5dff-43f5-b06c-f917cc7f75c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l"
Jul 15 17:40:10.127959 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.127925    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b495ff27-5dff-43f5-b06c-f917cc7f75c7-maas-api-tls\") pod \"maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l\" (UID: \"b495ff27-5dff-43f5-b06c-f917cc7f75c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l"
Jul 15 17:40:10.130313 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.130285    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b495ff27-5dff-43f5-b06c-f917cc7f75c7-maas-api-tls\") pod \"maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l\" (UID: \"b495ff27-5dff-43f5-b06c-f917cc7f75c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l"
Jul 15 17:40:10.334527 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.334492    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l"
Jul 15 17:40:10.678901 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.678878    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l"]
Jul 15 17:40:10.681208 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:40:10.681177    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb495ff27_5dff_43f5_b06c_f917cc7f75c7.slice/crio-9e17cd543f52779fdaca01398785b45bf296e4c542b7344125e0f08da58ba2e6 WatchSource:0}: Error finding container 9e17cd543f52779fdaca01398785b45bf296e4c542b7344125e0f08da58ba2e6: Status 404 returned error can't find the container with id 9e17cd543f52779fdaca01398785b45bf296e4c542b7344125e0f08da58ba2e6
Jul 15 17:40:10.817247 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.817223    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:40:10.934161 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.934058    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d18c08f7-1996-4d15-928a-925dd24b2dad-istiod-ca-cert\") pod \"d18c08f7-1996-4d15-928a-925dd24b2dad\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") "
Jul 15 17:40:10.934161 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.934138    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-workload-certs\") pod \"d18c08f7-1996-4d15-928a-925dd24b2dad\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") "
Jul 15 17:40:10.934161 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.934156    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-envoy\") pod \"d18c08f7-1996-4d15-928a-925dd24b2dad\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") "
Jul 15 17:40:10.934459 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.934186    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-token\") pod \"d18c08f7-1996-4d15-928a-925dd24b2dad\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") "
Jul 15 17:40:10.934459 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.934248    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sx64b\" (UniqueName: \"kubernetes.io/projected/d18c08f7-1996-4d15-928a-925dd24b2dad-kube-api-access-sx64b\") pod \"d18c08f7-1996-4d15-928a-925dd24b2dad\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") "
Jul 15 17:40:10.934459 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.934274    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-podinfo\") pod \"d18c08f7-1996-4d15-928a-925dd24b2dad\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") "
Jul 15 17:40:10.934459 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.934303    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-workload-socket\") pod \"d18c08f7-1996-4d15-928a-925dd24b2dad\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") "
Jul 15 17:40:10.934459 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.934340    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-credential-socket\") pod \"d18c08f7-1996-4d15-928a-925dd24b2dad\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") "
Jul 15 17:40:10.934459 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.934381    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-data\") pod \"d18c08f7-1996-4d15-928a-925dd24b2dad\" (UID: \"d18c08f7-1996-4d15-928a-925dd24b2dad\") "
Jul 15 17:40:10.934795 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.934465    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d18c08f7-1996-4d15-928a-925dd24b2dad" (UID: "d18c08f7-1996-4d15-928a-925dd24b2dad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:40:10.934795 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.934585    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d18c08f7-1996-4d15-928a-925dd24b2dad" (UID: "d18c08f7-1996-4d15-928a-925dd24b2dad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:40:10.934795 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.934602    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d18c08f7-1996-4d15-928a-925dd24b2dad-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d18c08f7-1996-4d15-928a-925dd24b2dad" (UID: "d18c08f7-1996-4d15-928a-925dd24b2dad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:40:10.934795 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.934787    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d18c08f7-1996-4d15-928a-925dd24b2dad" (UID: "d18c08f7-1996-4d15-928a-925dd24b2dad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:40:10.934994 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.934884    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-credential-socket\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:40:10.934994 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.934902    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d18c08f7-1996-4d15-928a-925dd24b2dad-istiod-ca-cert\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:40:10.934994 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.934915    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-workload-certs\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:40:10.934994 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.934927    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-workload-socket\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:40:10.935429 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.935396    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-data" (OuterVolumeSpecName: "istio-data") pod "d18c08f7-1996-4d15-928a-925dd24b2dad" (UID: "d18c08f7-1996-4d15-928a-925dd24b2dad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:40:10.936533 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.936503    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d18c08f7-1996-4d15-928a-925dd24b2dad" (UID: "d18c08f7-1996-4d15-928a-925dd24b2dad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:40:10.936641 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.936550    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d18c08f7-1996-4d15-928a-925dd24b2dad" (UID: "d18c08f7-1996-4d15-928a-925dd24b2dad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 17:40:10.936808 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.936790    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-token" (OuterVolumeSpecName: "istio-token") pod "d18c08f7-1996-4d15-928a-925dd24b2dad" (UID: "d18c08f7-1996-4d15-928a-925dd24b2dad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:40:10.936859 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:10.936810    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d18c08f7-1996-4d15-928a-925dd24b2dad-kube-api-access-sx64b" (OuterVolumeSpecName: "kube-api-access-sx64b") pod "d18c08f7-1996-4d15-928a-925dd24b2dad" (UID: "d18c08f7-1996-4d15-928a-925dd24b2dad"). InnerVolumeSpecName "kube-api-access-sx64b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:40:11.036011 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:11.035983    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sx64b\" (UniqueName: \"kubernetes.io/projected/d18c08f7-1996-4d15-928a-925dd24b2dad-kube-api-access-sx64b\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:40:11.036011 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:11.036009    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-podinfo\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:40:11.036011 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:11.036019    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-data\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:40:11.036223 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:11.036026    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-envoy\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:40:11.036223 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:11.036035    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d18c08f7-1996-4d15-928a-925dd24b2dad-istio-token\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:40:11.487227 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:11.487146    2575 generic.go:356] "Generic (PLEG): container finished" podID="d18c08f7-1996-4d15-928a-925dd24b2dad" containerID="e8d38b1e5f1adcdaf8d0cd8c10db4b5adb77b68febda894f35f79175530db631" exitCode=0
Jul 15 17:40:11.487227 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:11.487208    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"
Jul 15 17:40:11.487456 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:11.487225    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5" event={"ID":"d18c08f7-1996-4d15-928a-925dd24b2dad","Type":"ContainerDied","Data":"e8d38b1e5f1adcdaf8d0cd8c10db4b5adb77b68febda894f35f79175530db631"}
Jul 15 17:40:11.487456 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:11.487279    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5" event={"ID":"d18c08f7-1996-4d15-928a-925dd24b2dad","Type":"ContainerDied","Data":"c838f5512e2ad06518be8388755b2435a6b8c1e72666d41799e65b3d8b19bc1a"}
Jul 15 17:40:11.487456 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:11.487304    2575 scope.go:117] "RemoveContainer" containerID="e8d38b1e5f1adcdaf8d0cd8c10db4b5adb77b68febda894f35f79175530db631"
Jul 15 17:40:11.489303 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:11.489218    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l" event={"ID":"b495ff27-5dff-43f5-b06c-f917cc7f75c7","Type":"ContainerStarted","Data":"4a287dad6a72bf823b8014852defefaad5ed5281b1c6448c32b3378792dea120"}
Jul 15 17:40:11.489303 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:11.489266    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l" event={"ID":"b495ff27-5dff-43f5-b06c-f917cc7f75c7","Type":"ContainerStarted","Data":"9e17cd543f52779fdaca01398785b45bf296e4c542b7344125e0f08da58ba2e6"}
Jul 15 17:40:11.489488 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:11.489373    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l"
Jul 15 17:40:11.496377 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:11.496359    2575 scope.go:117] "RemoveContainer" containerID="dd654189e9a9332f1bb35a78e07e60d2c7855e92036203affc91e6e6dfe27afb"
Jul 15 17:40:11.504438 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:11.504421    2575 scope.go:117] "RemoveContainer" containerID="e8d38b1e5f1adcdaf8d0cd8c10db4b5adb77b68febda894f35f79175530db631"
Jul 15 17:40:11.504696 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:40:11.504675    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8d38b1e5f1adcdaf8d0cd8c10db4b5adb77b68febda894f35f79175530db631\": container with ID starting with e8d38b1e5f1adcdaf8d0cd8c10db4b5adb77b68febda894f35f79175530db631 not found: ID does not exist" containerID="e8d38b1e5f1adcdaf8d0cd8c10db4b5adb77b68febda894f35f79175530db631"
Jul 15 17:40:11.504782 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:11.504708    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8d38b1e5f1adcdaf8d0cd8c10db4b5adb77b68febda894f35f79175530db631"} err="failed to get container status \"e8d38b1e5f1adcdaf8d0cd8c10db4b5adb77b68febda894f35f79175530db631\": rpc error: code = NotFound desc = could not find container \"e8d38b1e5f1adcdaf8d0cd8c10db4b5adb77b68febda894f35f79175530db631\": container with ID starting with e8d38b1e5f1adcdaf8d0cd8c10db4b5adb77b68febda894f35f79175530db631 not found: ID does not exist"
Jul 15 17:40:11.504782 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:11.504733    2575 scope.go:117] "RemoveContainer" containerID="dd654189e9a9332f1bb35a78e07e60d2c7855e92036203affc91e6e6dfe27afb"
Jul 15 17:40:11.504977 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:40:11.504959    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd654189e9a9332f1bb35a78e07e60d2c7855e92036203affc91e6e6dfe27afb\": container with ID starting with dd654189e9a9332f1bb35a78e07e60d2c7855e92036203affc91e6e6dfe27afb not found: ID does not exist" containerID="dd654189e9a9332f1bb35a78e07e60d2c7855e92036203affc91e6e6dfe27afb"
Jul 15 17:40:11.505016 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:11.504983    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd654189e9a9332f1bb35a78e07e60d2c7855e92036203affc91e6e6dfe27afb"} err="failed to get container status \"dd654189e9a9332f1bb35a78e07e60d2c7855e92036203affc91e6e6dfe27afb\": rpc error: code = NotFound desc = could not find container \"dd654189e9a9332f1bb35a78e07e60d2c7855e92036203affc91e6e6dfe27afb\": container with ID starting with dd654189e9a9332f1bb35a78e07e60d2c7855e92036203affc91e6e6dfe27afb not found: ID does not exist"
Jul 15 17:40:11.511472 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:11.511427    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l" podStartSLOduration=2.028024747 podStartE2EDuration="2.511415705s" totalImagesPullingTime="483.390958ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:40:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:40:11.509152372 +0000 UTC m=+2627.526565498" watchObservedRunningTime="2026-07-15 17:40:11.511415705 +0000 UTC m=+2627.528828801"
Jul 15 17:40:11.524622 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:11.524595    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"]
Jul 15 17:40:11.530752 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:11.530730    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-1da9d9-openshift-default-76fcc5997c-mnrx5"]
Jul 15 17:40:12.525989 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:12.525957    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d18c08f7-1996-4d15-928a-925dd24b2dad" path="/var/lib/kubelet/pods/d18c08f7-1996-4d15-928a-925dd24b2dad/volumes"
Jul 15 17:40:14.391519 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.391488    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"]
Jul 15 17:40:14.391898 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.391871    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d18c08f7-1996-4d15-928a-925dd24b2dad" containerName="istio-proxy"
Jul 15 17:40:14.391898 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.391883    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d18c08f7-1996-4d15-928a-925dd24b2dad" containerName="istio-proxy"
Jul 15 17:40:14.391975 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.391955    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d18c08f7-1996-4d15-928a-925dd24b2dad" containerName="istio-proxy"
Jul 15 17:40:14.391975 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.391965    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d18c08f7-1996-4d15-928a-925dd24b2dad" containerName="istio-proxy"
Jul 15 17:40:14.395105 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.395086    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.405833 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.405808    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-81a039-openshift-default-dockercfg-zg4lq\""
Jul 15 17:40:14.434470 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.434432    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"]
Jul 15 17:40:14.466927 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.466875    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-credential-socket\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.467124 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.467101    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-token\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.467201 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.467159    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istiod-ca-cert\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.467265 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.467214    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-envoy\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.467265 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.467253    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-data\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.467349 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.467331    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-workload-socket\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.467404 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.467391    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5z55\" (UniqueName: \"kubernetes.io/projected/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-kube-api-access-t5z55\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.467442 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.467427    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-workload-certs\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.467488 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.467461    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-podinfo\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.568145 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.568102    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5z55\" (UniqueName: \"kubernetes.io/projected/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-kube-api-access-t5z55\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.568318 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.568155    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-workload-certs\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.568318 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.568195    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-podinfo\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.568318 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.568264    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-credential-socket\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.568318 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.568280    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-token\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.568599 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.568322    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istiod-ca-cert\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.568599 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.568463    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-envoy\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.568599 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.568514    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-data\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.568599 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.568575    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-workload-socket\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.568802 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.568656    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-workload-certs\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.568802 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.568711    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-credential-socket\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.568935 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.568913    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-data\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.569022 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.568972    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-workload-socket\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.569111 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.569089    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istiod-ca-cert\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.570832 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.570811    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-envoy\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.570960 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.570943    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-podinfo\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.578685 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.578666    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5z55\" (UniqueName: \"kubernetes.io/projected/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-kube-api-access-t5z55\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.582663 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.582644    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-token\") pod \"e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") " pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.713652 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.713574    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:14.851637 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.851569    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"]
Jul 15 17:40:14.853525 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:40:14.853501    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0149e98f_7f35_4f1c_b29c_f2bbab3f8006.slice/crio-b2bbe88fea00358851258a8a098c1a4fbaf70a3edd1ee671479faa8f3c22e1d9 WatchSource:0}: Error finding container b2bbe88fea00358851258a8a098c1a4fbaf70a3edd1ee671479faa8f3c22e1d9: Status 404 returned error can't find the container with id b2bbe88fea00358851258a8a098c1a4fbaf70a3edd1ee671479faa8f3c22e1d9
Jul 15 17:40:14.855980 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.855949    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:40:14.856090 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.856024    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:40:14.856090 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:14.856055    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:40:15.511678 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:15.511644    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2" event={"ID":"0149e98f-7f35-4f1c-b29c-f2bbab3f8006","Type":"ContainerStarted","Data":"61b942c27da89ab5a8d12773629e560853715af1fbf905d79a76893c0adc2bd0"}
Jul 15 17:40:15.512068 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:15.511695    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2" event={"ID":"0149e98f-7f35-4f1c-b29c-f2bbab3f8006","Type":"ContainerStarted","Data":"b2bbe88fea00358851258a8a098c1a4fbaf70a3edd1ee671479faa8f3c22e1d9"}
Jul 15 17:40:15.533025 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:15.532979    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2" podStartSLOduration=1.532964182 podStartE2EDuration="1.532964182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:40:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:40:15.531538361 +0000 UTC m=+2631.548951457" watchObservedRunningTime="2026-07-15 17:40:15.532964182 +0000 UTC m=+2631.550377277"
Jul 15 17:40:15.714449 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:15.714421    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:15.719340 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:15.719317    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:16.516866 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:16.516840    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:16.517681 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:16.517659    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:17.499934 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:17.499907    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l"
Jul 15 17:40:17.532729 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:17.532693    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz"]
Jul 15 17:40:17.533247 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:17.533227    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d18c08f7-1996-4d15-928a-925dd24b2dad" containerName="istio-proxy"
Jul 15 17:40:17.533338 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:17.533250    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d18c08f7-1996-4d15-928a-925dd24b2dad" containerName="istio-proxy"
Jul 15 17:40:17.540377 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:17.540347    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz"
Jul 15 17:40:17.542911 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:17.542887    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-81a039-serving-cert\""
Jul 15 17:40:17.546551 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:17.546519    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz"]
Jul 15 17:40:17.698947 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:17.698913    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b4b29d3-597a-4bf1-b5f0-4a6e15181f29-openshift-service-ca\") pod \"maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz\" (UID: \"3b4b29d3-597a-4bf1-b5f0-4a6e15181f29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz"
Jul 15 17:40:17.699119 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:17.698966    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b4b29d3-597a-4bf1-b5f0-4a6e15181f29-maas-api-tls\") pod \"maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz\" (UID: \"3b4b29d3-597a-4bf1-b5f0-4a6e15181f29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz"
Jul 15 17:40:17.699188 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:17.699117    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz64f\" (UniqueName: \"kubernetes.io/projected/3b4b29d3-597a-4bf1-b5f0-4a6e15181f29-kube-api-access-cz64f\") pod \"maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz\" (UID: \"3b4b29d3-597a-4bf1-b5f0-4a6e15181f29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz"
Jul 15 17:40:17.800146 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:17.800046    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b4b29d3-597a-4bf1-b5f0-4a6e15181f29-openshift-service-ca\") pod \"maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz\" (UID: \"3b4b29d3-597a-4bf1-b5f0-4a6e15181f29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz"
Jul 15 17:40:17.800146 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:17.800103    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b4b29d3-597a-4bf1-b5f0-4a6e15181f29-maas-api-tls\") pod \"maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz\" (UID: \"3b4b29d3-597a-4bf1-b5f0-4a6e15181f29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz"
Jul 15 17:40:17.800146 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:17.800132    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz64f\" (UniqueName: \"kubernetes.io/projected/3b4b29d3-597a-4bf1-b5f0-4a6e15181f29-kube-api-access-cz64f\") pod \"maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz\" (UID: \"3b4b29d3-597a-4bf1-b5f0-4a6e15181f29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz"
Jul 15 17:40:17.800426 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:40:17.800248    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-serving-cert: secret "maas-api-e2e-shared-b-81a039-serving-cert" not found
Jul 15 17:40:17.800426 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:40:17.800318    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b4b29d3-597a-4bf1-b5f0-4a6e15181f29-maas-api-tls podName:3b4b29d3-597a-4bf1-b5f0-4a6e15181f29 nodeName:}" failed. No retries permitted until 2026-07-15 17:40:18.300300071 +0000 UTC m=+2634.317713144 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3b4b29d3-597a-4bf1-b5f0-4a6e15181f29-maas-api-tls") pod "maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz" (UID: "3b4b29d3-597a-4bf1-b5f0-4a6e15181f29") : secret "maas-api-e2e-shared-b-81a039-serving-cert" not found
Jul 15 17:40:17.800695 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:17.800678    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b4b29d3-597a-4bf1-b5f0-4a6e15181f29-openshift-service-ca\") pod \"maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz\" (UID: \"3b4b29d3-597a-4bf1-b5f0-4a6e15181f29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz"
Jul 15 17:40:17.810266 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:17.810242    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz64f\" (UniqueName: \"kubernetes.io/projected/3b4b29d3-597a-4bf1-b5f0-4a6e15181f29-kube-api-access-cz64f\") pod \"maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz\" (UID: \"3b4b29d3-597a-4bf1-b5f0-4a6e15181f29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz"
Jul 15 17:40:18.308367 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:18.308326    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b4b29d3-597a-4bf1-b5f0-4a6e15181f29-maas-api-tls\") pod \"maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz\" (UID: \"3b4b29d3-597a-4bf1-b5f0-4a6e15181f29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz"
Jul 15 17:40:18.310722 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:18.310700    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b4b29d3-597a-4bf1-b5f0-4a6e15181f29-maas-api-tls\") pod \"maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz\" (UID: \"3b4b29d3-597a-4bf1-b5f0-4a6e15181f29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz"
Jul 15 17:40:18.454986 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:18.454954    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz"
Jul 15 17:40:18.597112 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:18.597089    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz"]
Jul 15 17:40:18.599278 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:40:18.599253    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b4b29d3_597a_4bf1_b5f0_4a6e15181f29.slice/crio-524d80249d0c5e522a244554713b5cb04dfc45f9ccf5fe315b7d4031c661f5e6 WatchSource:0}: Error finding container 524d80249d0c5e522a244554713b5cb04dfc45f9ccf5fe315b7d4031c661f5e6: Status 404 returned error can't find the container with id 524d80249d0c5e522a244554713b5cb04dfc45f9ccf5fe315b7d4031c661f5e6
Jul 15 17:40:19.537013 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:19.536963    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz" event={"ID":"3b4b29d3-597a-4bf1-b5f0-4a6e15181f29","Type":"ContainerStarted","Data":"5beb06acfa5a12c4f087d3eb99aea100a49a2437fdccdc57d14281a8933a80d5"}
Jul 15 17:40:19.537176 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:19.537027    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz" event={"ID":"3b4b29d3-597a-4bf1-b5f0-4a6e15181f29","Type":"ContainerStarted","Data":"524d80249d0c5e522a244554713b5cb04dfc45f9ccf5fe315b7d4031c661f5e6"}
Jul 15 17:40:19.537176 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:19.537060    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz"
Jul 15 17:40:19.558649 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:19.558549    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz" podStartSLOduration=1.8703517029999999 podStartE2EDuration="2.558535089s" totalImagesPullingTime="688.183386ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:40:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:40:19.556627844 +0000 UTC m=+2635.574041143" watchObservedRunningTime="2026-07-15 17:40:19.558535089 +0000 UTC m=+2635.575948184"
Jul 15 17:40:25.552863 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:25.552821    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz"
Jul 15 17:40:29.591446 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:29.591413    2575 generic.go:356] "Generic (PLEG): container finished" podID="0149e98f-7f35-4f1c-b29c-f2bbab3f8006" containerID="61b942c27da89ab5a8d12773629e560853715af1fbf905d79a76893c0adc2bd0" exitCode=0
Jul 15 17:40:29.591885 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:29.591490    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2" event={"ID":"0149e98f-7f35-4f1c-b29c-f2bbab3f8006","Type":"ContainerDied","Data":"61b942c27da89ab5a8d12773629e560853715af1fbf905d79a76893c0adc2bd0"}
Jul 15 17:40:29.591929 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:29.591897    2575 scope.go:117] "RemoveContainer" containerID="61b942c27da89ab5a8d12773629e560853715af1fbf905d79a76893c0adc2bd0"
Jul 15 17:40:29.592911 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:29.592883    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:40:29.592983 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:29.592966    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:40:29.593057 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:29.593041    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 17:40:30.599003 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:30.598965    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2" event={"ID":"0149e98f-7f35-4f1c-b29c-f2bbab3f8006","Type":"ContainerStarted","Data":"25dac47eba6c2dec07444e653ce4be5973d2a24baff5bb41c21a6acdbe600b8e"}
Jul 15 17:40:30.713905 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:30.713872    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:30.715056 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:30.715032    2575 patch_prober.go:28] interesting pod/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 15 17:40:30.715169 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:30.715092    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2" podUID="0149e98f-7f35-4f1c-b29c-f2bbab3f8006" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 15 17:40:31.517395 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:31.517349    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:31.714510 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:31.714473    2575 patch_prober.go:28] interesting pod/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 15 17:40:31.714922 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:31.714549    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2" podUID="0149e98f-7f35-4f1c-b29c-f2bbab3f8006" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 15 17:40:32.714036 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:32.713996    2575 patch_prober.go:28] interesting pod/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 15 17:40:32.714214 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:32.714057    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2" podUID="0149e98f-7f35-4f1c-b29c-f2bbab3f8006" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 15 17:40:33.714883 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:33.714845    2575 patch_prober.go:28] interesting pod/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 15 17:40:33.715281 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:33.714905    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2" podUID="0149e98f-7f35-4f1c-b29c-f2bbab3f8006" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 15 17:40:34.718146 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:34.718118    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:40:34.719151 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:40:34.719134    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:41:21.115722 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.115687    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"]
Jul 15 17:41:21.119293 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.119273    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:21.122214 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.122190    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-81a039\"/\"kube-root-ca.crt\""
Jul 15 17:41:21.122334 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.122219    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-81a039\"/\"openshift-service-ca.crt\""
Jul 15 17:41:21.123299 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.123268    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-81a039\"/\"auth-test-model-81a039-kserve-self-signed-certs\""
Jul 15 17:41:21.123299 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.123275    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-81a039\"/\"default-dockercfg-c4jvt\""
Jul 15 17:41:21.128671 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.128650    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"]
Jul 15 17:41:21.200222 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.200192    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvqhw\" (UniqueName: \"kubernetes.io/projected/a2e57a83-181f-41c7-b738-5c83f3d9ded0-kube-api-access-fvqhw\") pod \"auth-test-model-81a039-kserve-64cb6db467-cqknp\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") " pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:21.200366 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.200236    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-dshm\") pod \"auth-test-model-81a039-kserve-64cb6db467-cqknp\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") " pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:21.200366 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.200263    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-model-cache\") pod \"auth-test-model-81a039-kserve-64cb6db467-cqknp\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") " pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:21.200366 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.200356    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2e57a83-181f-41c7-b738-5c83f3d9ded0-tls-certs\") pod \"auth-test-model-81a039-kserve-64cb6db467-cqknp\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") " pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:21.200490 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.200388    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-home\") pod \"auth-test-model-81a039-kserve-64cb6db467-cqknp\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") " pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:21.200490 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.200411    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-tmp-dir\") pod \"auth-test-model-81a039-kserve-64cb6db467-cqknp\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") " pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:21.301379 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.301342    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2e57a83-181f-41c7-b738-5c83f3d9ded0-tls-certs\") pod \"auth-test-model-81a039-kserve-64cb6db467-cqknp\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") " pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:21.301551 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.301390    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-home\") pod \"auth-test-model-81a039-kserve-64cb6db467-cqknp\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") " pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:21.301551 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.301412    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-tmp-dir\") pod \"auth-test-model-81a039-kserve-64cb6db467-cqknp\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") " pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:21.301551 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.301436    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvqhw\" (UniqueName: \"kubernetes.io/projected/a2e57a83-181f-41c7-b738-5c83f3d9ded0-kube-api-access-fvqhw\") pod \"auth-test-model-81a039-kserve-64cb6db467-cqknp\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") " pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:21.301551 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.301489    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-dshm\") pod \"auth-test-model-81a039-kserve-64cb6db467-cqknp\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") " pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:21.301551 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.301522    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-model-cache\") pod \"auth-test-model-81a039-kserve-64cb6db467-cqknp\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") " pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:21.301829 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.301805    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-home\") pod \"auth-test-model-81a039-kserve-64cb6db467-cqknp\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") " pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:21.301886 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.301842    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-tmp-dir\") pod \"auth-test-model-81a039-kserve-64cb6db467-cqknp\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") " pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:21.301966 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.301946    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-model-cache\") pod \"auth-test-model-81a039-kserve-64cb6db467-cqknp\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") " pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:21.303750 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.303732    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-dshm\") pod \"auth-test-model-81a039-kserve-64cb6db467-cqknp\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") " pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:21.304029 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.304010    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2e57a83-181f-41c7-b738-5c83f3d9ded0-tls-certs\") pod \"auth-test-model-81a039-kserve-64cb6db467-cqknp\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") " pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:21.312767 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.312746    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvqhw\" (UniqueName: \"kubernetes.io/projected/a2e57a83-181f-41c7-b738-5c83f3d9ded0-kube-api-access-fvqhw\") pod \"auth-test-model-81a039-kserve-64cb6db467-cqknp\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") " pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:21.429354 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.429274    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:21.573890 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.573863    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"]
Jul 15 17:41:21.576355 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:41:21.576320    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2e57a83_181f_41c7_b738_5c83f3d9ded0.slice/crio-1fb7a8e888fa351fa0cbc92966dd01849f1d84c02d7168571c7c745fea1b47c3 WatchSource:0}: Error finding container 1fb7a8e888fa351fa0cbc92966dd01849f1d84c02d7168571c7c745fea1b47c3: Status 404 returned error can't find the container with id 1fb7a8e888fa351fa0cbc92966dd01849f1d84c02d7168571c7c745fea1b47c3
Jul 15 17:41:21.852177 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.852083    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp" event={"ID":"a2e57a83-181f-41c7-b738-5c83f3d9ded0","Type":"ContainerStarted","Data":"1fba469f4e283723fe36678c5caf63535bcba1119b6d19d8c465765949cb0ab3"}
Jul 15 17:41:21.852177 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.852135    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp" event={"ID":"a2e57a83-181f-41c7-b738-5c83f3d9ded0","Type":"ContainerStarted","Data":"1fb7a8e888fa351fa0cbc92966dd01849f1d84c02d7168571c7c745fea1b47c3"}
Jul 15 17:41:21.872768 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:21.872713    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp" podStartSLOduration=0.872699017 podStartE2EDuration="872.699017ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:41:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:41:21.870778142 +0000 UTC m=+2697.888191303" watchObservedRunningTime="2026-07-15 17:41:21.872699017 +0000 UTC m=+2697.890112112"
Jul 15 17:41:24.598042 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:24.598015    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 17:41:24.606164 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:24.606142    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 17:41:31.429524 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:31.429493    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:31.430143 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:31.429605    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:31.441963 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:31.441940    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:41:31.913907 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:41:31.913875    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:43:24.898469 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:24.898433    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"]
Jul 15 17:43:38.813576 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:38.813532    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"]
Jul 15 17:43:38.816788 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:38.816772    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:38.819407 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:38.819386    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3d6c37\"/\"default-dockercfg-xds9s\""
Jul 15 17:43:38.819543 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:38.819526    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3d6c37\"/\"openshift-service-ca.crt\""
Jul 15 17:43:38.820420 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:38.820405    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3d6c37\"/\"kube-root-ca.crt\""
Jul 15 17:43:38.820486 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:38.820463    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3d6c37\"/\"sub-test-model-3d6c37-kserve-self-signed-certs\""
Jul 15 17:43:38.828630 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:38.828609    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"]
Jul 15 17:43:38.926613 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:38.926586    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-home\") pod \"sub-test-model-3d6c37-kserve-66cb56c-qkt5b\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") " pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:38.926789 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:38.926633    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-tmp-dir\") pod \"sub-test-model-3d6c37-kserve-66cb56c-qkt5b\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") " pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:38.926789 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:38.926654    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-dshm\") pod \"sub-test-model-3d6c37-kserve-66cb56c-qkt5b\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") " pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:38.926789 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:38.926748    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prtff\" (UniqueName: \"kubernetes.io/projected/71b30ffe-0950-4e90-ba03-cf43318fce22-kube-api-access-prtff\") pod \"sub-test-model-3d6c37-kserve-66cb56c-qkt5b\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") " pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:38.926931 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:38.926795    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-model-cache\") pod \"sub-test-model-3d6c37-kserve-66cb56c-qkt5b\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") " pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:38.926931 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:38.926831    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71b30ffe-0950-4e90-ba03-cf43318fce22-tls-certs\") pod \"sub-test-model-3d6c37-kserve-66cb56c-qkt5b\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") " pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:39.028137 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:39.028096    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-home\") pod \"sub-test-model-3d6c37-kserve-66cb56c-qkt5b\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") " pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:39.028313 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:39.028155    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-tmp-dir\") pod \"sub-test-model-3d6c37-kserve-66cb56c-qkt5b\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") " pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:39.028313 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:39.028183    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-dshm\") pod \"sub-test-model-3d6c37-kserve-66cb56c-qkt5b\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") " pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:39.028313 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:39.028210    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prtff\" (UniqueName: \"kubernetes.io/projected/71b30ffe-0950-4e90-ba03-cf43318fce22-kube-api-access-prtff\") pod \"sub-test-model-3d6c37-kserve-66cb56c-qkt5b\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") " pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:39.028313 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:39.028226    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-model-cache\") pod \"sub-test-model-3d6c37-kserve-66cb56c-qkt5b\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") " pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:39.028313 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:39.028252    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71b30ffe-0950-4e90-ba03-cf43318fce22-tls-certs\") pod \"sub-test-model-3d6c37-kserve-66cb56c-qkt5b\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") " pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:39.028604 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:39.028550    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-home\") pod \"sub-test-model-3d6c37-kserve-66cb56c-qkt5b\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") " pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:39.028604 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:39.028589    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-tmp-dir\") pod \"sub-test-model-3d6c37-kserve-66cb56c-qkt5b\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") " pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:39.028734 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:39.028677    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-model-cache\") pod \"sub-test-model-3d6c37-kserve-66cb56c-qkt5b\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") " pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:39.030578 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:39.030543    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-dshm\") pod \"sub-test-model-3d6c37-kserve-66cb56c-qkt5b\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") " pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:39.030660 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:39.030645    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71b30ffe-0950-4e90-ba03-cf43318fce22-tls-certs\") pod \"sub-test-model-3d6c37-kserve-66cb56c-qkt5b\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") " pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:39.036125 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:39.036103    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prtff\" (UniqueName: \"kubernetes.io/projected/71b30ffe-0950-4e90-ba03-cf43318fce22-kube-api-access-prtff\") pod \"sub-test-model-3d6c37-kserve-66cb56c-qkt5b\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") " pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:39.125979 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:39.125933    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:39.259475 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:39.259446    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"]
Jul 15 17:43:39.262627 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:43:39.262600    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71b30ffe_0950_4e90_ba03_cf43318fce22.slice/crio-d746ff582408e62793d4d0c8f2535b2508278aaff89b74314a542ec6abaa45d9 WatchSource:0}: Error finding container d746ff582408e62793d4d0c8f2535b2508278aaff89b74314a542ec6abaa45d9: Status 404 returned error can't find the container with id d746ff582408e62793d4d0c8f2535b2508278aaff89b74314a542ec6abaa45d9
Jul 15 17:43:39.536641 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:39.536524    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b" event={"ID":"71b30ffe-0950-4e90-ba03-cf43318fce22","Type":"ContainerStarted","Data":"8c489b22a9315c19edc3cdcbc3573066522c0d0723d3b5cc042940fdb7a8ce3c"}
Jul 15 17:43:39.536641 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:39.536585    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b" event={"ID":"71b30ffe-0950-4e90-ba03-cf43318fce22","Type":"ContainerStarted","Data":"d746ff582408e62793d4d0c8f2535b2508278aaff89b74314a542ec6abaa45d9"}
Jul 15 17:43:39.555348 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:39.555295    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b" podStartSLOduration=1.5552783780000001 podStartE2EDuration="1.555278378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:43:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:43:39.555092687 +0000 UTC m=+2835.572505784" watchObservedRunningTime="2026-07-15 17:43:39.555278378 +0000 UTC m=+2835.572691475"
Jul 15 17:43:39.918674 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:39.918631    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp" podUID="a2e57a83-181f-41c7-b738-5c83f3d9ded0" containerName="main" containerID="cri-o://1fba469f4e283723fe36678c5caf63535bcba1119b6d19d8c465765949cb0ab3" gracePeriod=45
Jul 15 17:43:40.165183 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.165158    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:43:40.239535 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.239461    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2e57a83-181f-41c7-b738-5c83f3d9ded0-tls-certs\") pod \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") "
Jul 15 17:43:40.239535 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.239496    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-model-cache\") pod \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") "
Jul 15 17:43:40.239771 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.239579    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-dshm\") pod \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") "
Jul 15 17:43:40.239771 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.239609    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-home\") pod \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") "
Jul 15 17:43:40.239771 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.239674    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-tmp-dir\") pod \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") "
Jul 15 17:43:40.239771 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.239704    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvqhw\" (UniqueName: \"kubernetes.io/projected/a2e57a83-181f-41c7-b738-5c83f3d9ded0-kube-api-access-fvqhw\") pod \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\" (UID: \"a2e57a83-181f-41c7-b738-5c83f3d9ded0\") "
Jul 15 17:43:40.239998 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.239861    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-model-cache" (OuterVolumeSpecName: "model-cache") pod "a2e57a83-181f-41c7-b738-5c83f3d9ded0" (UID: "a2e57a83-181f-41c7-b738-5c83f3d9ded0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:43:40.239998 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.239908    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-home" (OuterVolumeSpecName: "home") pod "a2e57a83-181f-41c7-b738-5c83f3d9ded0" (UID: "a2e57a83-181f-41c7-b738-5c83f3d9ded0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:43:40.239998 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.239918    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a2e57a83-181f-41c7-b738-5c83f3d9ded0" (UID: "a2e57a83-181f-41c7-b738-5c83f3d9ded0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:43:40.240144 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.240078    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-model-cache\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:43:40.240144 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.240094    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-home\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:43:40.240144 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.240107    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-tmp-dir\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:43:40.241768 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.241745    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-dshm" (OuterVolumeSpecName: "dshm") pod "a2e57a83-181f-41c7-b738-5c83f3d9ded0" (UID: "a2e57a83-181f-41c7-b738-5c83f3d9ded0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:43:40.241886 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.241800    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2e57a83-181f-41c7-b738-5c83f3d9ded0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a2e57a83-181f-41c7-b738-5c83f3d9ded0" (UID: "a2e57a83-181f-41c7-b738-5c83f3d9ded0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:43:40.241933 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.241882    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2e57a83-181f-41c7-b738-5c83f3d9ded0-kube-api-access-fvqhw" (OuterVolumeSpecName: "kube-api-access-fvqhw") pod "a2e57a83-181f-41c7-b738-5c83f3d9ded0" (UID: "a2e57a83-181f-41c7-b738-5c83f3d9ded0"). InnerVolumeSpecName "kube-api-access-fvqhw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:43:40.340632 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.340607    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2e57a83-181f-41c7-b738-5c83f3d9ded0-dshm\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:43:40.340632 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.340629    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvqhw\" (UniqueName: \"kubernetes.io/projected/a2e57a83-181f-41c7-b738-5c83f3d9ded0-kube-api-access-fvqhw\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:43:40.340785 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.340640    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2e57a83-181f-41c7-b738-5c83f3d9ded0-tls-certs\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:43:40.543611 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.543578    2575 generic.go:356] "Generic (PLEG): container finished" podID="a2e57a83-181f-41c7-b738-5c83f3d9ded0" containerID="1fba469f4e283723fe36678c5caf63535bcba1119b6d19d8c465765949cb0ab3" exitCode=0
Jul 15 17:43:40.543765 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.543642    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"
Jul 15 17:43:40.543765 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.543661    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp" event={"ID":"a2e57a83-181f-41c7-b738-5c83f3d9ded0","Type":"ContainerDied","Data":"1fba469f4e283723fe36678c5caf63535bcba1119b6d19d8c465765949cb0ab3"}
Jul 15 17:43:40.543765 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.543712    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp" event={"ID":"a2e57a83-181f-41c7-b738-5c83f3d9ded0","Type":"ContainerDied","Data":"1fb7a8e888fa351fa0cbc92966dd01849f1d84c02d7168571c7c745fea1b47c3"}
Jul 15 17:43:40.543765 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.543738    2575 scope.go:117] "RemoveContainer" containerID="1fba469f4e283723fe36678c5caf63535bcba1119b6d19d8c465765949cb0ab3"
Jul 15 17:43:40.552992 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.552972    2575 scope.go:117] "RemoveContainer" containerID="1fba469f4e283723fe36678c5caf63535bcba1119b6d19d8c465765949cb0ab3"
Jul 15 17:43:40.553242 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:43:40.553222    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fba469f4e283723fe36678c5caf63535bcba1119b6d19d8c465765949cb0ab3\": container with ID starting with 1fba469f4e283723fe36678c5caf63535bcba1119b6d19d8c465765949cb0ab3 not found: ID does not exist" containerID="1fba469f4e283723fe36678c5caf63535bcba1119b6d19d8c465765949cb0ab3"
Jul 15 17:43:40.553286 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.553250    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fba469f4e283723fe36678c5caf63535bcba1119b6d19d8c465765949cb0ab3"} err="failed to get container status \"1fba469f4e283723fe36678c5caf63535bcba1119b6d19d8c465765949cb0ab3\": rpc error: code = NotFound desc = could not find container \"1fba469f4e283723fe36678c5caf63535bcba1119b6d19d8c465765949cb0ab3\": container with ID starting with 1fba469f4e283723fe36678c5caf63535bcba1119b6d19d8c465765949cb0ab3 not found: ID does not exist"
Jul 15 17:43:40.561048 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.561026    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"]
Jul 15 17:43:40.566860 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:40.566841    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-81a039/auth-test-model-81a039-kserve-64cb6db467-cqknp"]
Jul 15 17:43:42.526031 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:42.525997    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2e57a83-181f-41c7-b738-5c83f3d9ded0" path="/var/lib/kubelet/pods/a2e57a83-181f-41c7-b738-5c83f3d9ded0/volumes"
Jul 15 17:43:49.126696 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:49.126661    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:49.127163 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:49.126707    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:49.138537 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:49.138513    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:49.600466 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:49.600383    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:43:58.099786 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.099752    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"]
Jul 15 17:43:58.100371 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.100290    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2e57a83-181f-41c7-b738-5c83f3d9ded0" containerName="main"
Jul 15 17:43:58.100371 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.100307    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2e57a83-181f-41c7-b738-5c83f3d9ded0" containerName="main"
Jul 15 17:43:58.100457 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.100395    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2e57a83-181f-41c7-b738-5c83f3d9ded0" containerName="main"
Jul 15 17:43:58.104657 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.104641    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:43:58.107512 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.107489    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-81a039\"/\"openshift-service-ca.crt\""
Jul 15 17:43:58.108966 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.108942    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-81a039\"/\"kube-root-ca.crt\""
Jul 15 17:43:58.109087 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.108997    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-81a039\"/\"default-dockercfg-c4jvt\""
Jul 15 17:43:58.109087 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.109012    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-81a039\"/\"sub-test-model-81a039-kserve-self-signed-certs\""
Jul 15 17:43:58.111478 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.111453    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"]
Jul 15 17:43:58.194580 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.194529    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-dshm\") pod \"sub-test-model-81a039-kserve-67b8b4bff5-zr224\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") " pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:43:58.194733 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.194600    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-home\") pod \"sub-test-model-81a039-kserve-67b8b4bff5-zr224\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") " pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:43:58.194733 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.194640    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13188fb4-3507-4565-92a1-6cbf7868b499-tls-certs\") pod \"sub-test-model-81a039-kserve-67b8b4bff5-zr224\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") " pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:43:58.194733 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.194707    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-tmp-dir\") pod \"sub-test-model-81a039-kserve-67b8b4bff5-zr224\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") " pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:43:58.194843 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.194740    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-model-cache\") pod \"sub-test-model-81a039-kserve-67b8b4bff5-zr224\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") " pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:43:58.194843 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.194784    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6lpp\" (UniqueName: \"kubernetes.io/projected/13188fb4-3507-4565-92a1-6cbf7868b499-kube-api-access-n6lpp\") pod \"sub-test-model-81a039-kserve-67b8b4bff5-zr224\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") " pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:43:58.295361 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.295327    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-model-cache\") pod \"sub-test-model-81a039-kserve-67b8b4bff5-zr224\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") " pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:43:58.295531 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.295364    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6lpp\" (UniqueName: \"kubernetes.io/projected/13188fb4-3507-4565-92a1-6cbf7868b499-kube-api-access-n6lpp\") pod \"sub-test-model-81a039-kserve-67b8b4bff5-zr224\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") " pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:43:58.295531 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.295406    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-dshm\") pod \"sub-test-model-81a039-kserve-67b8b4bff5-zr224\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") " pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:43:58.295531 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.295427    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-home\") pod \"sub-test-model-81a039-kserve-67b8b4bff5-zr224\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") " pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:43:58.295531 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.295461    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13188fb4-3507-4565-92a1-6cbf7868b499-tls-certs\") pod \"sub-test-model-81a039-kserve-67b8b4bff5-zr224\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") " pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:43:58.295531 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.295511    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-tmp-dir\") pod \"sub-test-model-81a039-kserve-67b8b4bff5-zr224\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") " pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:43:58.295840 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.295802    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-home\") pod \"sub-test-model-81a039-kserve-67b8b4bff5-zr224\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") " pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:43:58.295889 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.295803    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-model-cache\") pod \"sub-test-model-81a039-kserve-67b8b4bff5-zr224\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") " pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:43:58.295960 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.295942    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-tmp-dir\") pod \"sub-test-model-81a039-kserve-67b8b4bff5-zr224\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") " pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:43:58.297731 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.297701    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-dshm\") pod \"sub-test-model-81a039-kserve-67b8b4bff5-zr224\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") " pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:43:58.298024 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.297996    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13188fb4-3507-4565-92a1-6cbf7868b499-tls-certs\") pod \"sub-test-model-81a039-kserve-67b8b4bff5-zr224\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") " pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:43:58.303166 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.303148    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6lpp\" (UniqueName: \"kubernetes.io/projected/13188fb4-3507-4565-92a1-6cbf7868b499-kube-api-access-n6lpp\") pod \"sub-test-model-81a039-kserve-67b8b4bff5-zr224\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") " pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:43:58.415340 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.415307    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:43:58.556814 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.556791    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"]
Jul 15 17:43:58.558692 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:43:58.558665    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13188fb4_3507_4565_92a1_6cbf7868b499.slice/crio-30db0d8e024a563986e7b195a06f47214f631f0326538c8f18a5262604bf15ab WatchSource:0}: Error finding container 30db0d8e024a563986e7b195a06f47214f631f0326538c8f18a5262604bf15ab: Status 404 returned error can't find the container with id 30db0d8e024a563986e7b195a06f47214f631f0326538c8f18a5262604bf15ab
Jul 15 17:43:58.636209 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.636179    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224" event={"ID":"13188fb4-3507-4565-92a1-6cbf7868b499","Type":"ContainerStarted","Data":"1f880f40e9a7581bb35055beae0a6d54052007df810321e37300c691f205e399"}
Jul 15 17:43:58.636358 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.636218    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224" event={"ID":"13188fb4-3507-4565-92a1-6cbf7868b499","Type":"ContainerStarted","Data":"30db0d8e024a563986e7b195a06f47214f631f0326538c8f18a5262604bf15ab"}
Jul 15 17:43:58.658061 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:43:58.658012    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224" podStartSLOduration=0.657995627 podStartE2EDuration="657.995627ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:43:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:43:58.656466838 +0000 UTC m=+2854.673879945" watchObservedRunningTime="2026-07-15 17:43:58.657995627 +0000 UTC m=+2854.675408767"
Jul 15 17:44:08.415981 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:44:08.415941    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:44:08.415981 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:44:08.415991    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:44:08.428465 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:44:08.428437    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:44:08.693604 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:44:08.693510    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:44:51.169435 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:44:51.169402    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"]
Jul 15 17:44:52.085638 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:44:52.085607    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"]
Jul 15 17:45:06.189898 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.189856    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b" podUID="71b30ffe-0950-4e90-ba03-cf43318fce22" containerName="main" containerID="cri-o://8c489b22a9315c19edc3cdcbc3573066522c0d0723d3b5cc042940fdb7a8ce3c" gracePeriod=45
Jul 15 17:45:06.439704 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.439668    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:45:06.513971 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.513884    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-home\") pod \"71b30ffe-0950-4e90-ba03-cf43318fce22\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") "
Jul 15 17:45:06.513971 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.513922    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71b30ffe-0950-4e90-ba03-cf43318fce22-tls-certs\") pod \"71b30ffe-0950-4e90-ba03-cf43318fce22\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") "
Jul 15 17:45:06.513971 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.513960    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-tmp-dir\") pod \"71b30ffe-0950-4e90-ba03-cf43318fce22\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") "
Jul 15 17:45:06.514263 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.514008    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-dshm\") pod \"71b30ffe-0950-4e90-ba03-cf43318fce22\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") "
Jul 15 17:45:06.514263 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.514142    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prtff\" (UniqueName: \"kubernetes.io/projected/71b30ffe-0950-4e90-ba03-cf43318fce22-kube-api-access-prtff\") pod \"71b30ffe-0950-4e90-ba03-cf43318fce22\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") "
Jul 15 17:45:06.514263 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.514186    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-model-cache\") pod \"71b30ffe-0950-4e90-ba03-cf43318fce22\" (UID: \"71b30ffe-0950-4e90-ba03-cf43318fce22\") "
Jul 15 17:45:06.514263 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.514172    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-home" (OuterVolumeSpecName: "home") pod "71b30ffe-0950-4e90-ba03-cf43318fce22" (UID: "71b30ffe-0950-4e90-ba03-cf43318fce22"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:45:06.514484 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.514264    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "71b30ffe-0950-4e90-ba03-cf43318fce22" (UID: "71b30ffe-0950-4e90-ba03-cf43318fce22"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:45:06.514542 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.514472    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-model-cache" (OuterVolumeSpecName: "model-cache") pod "71b30ffe-0950-4e90-ba03-cf43318fce22" (UID: "71b30ffe-0950-4e90-ba03-cf43318fce22"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:45:06.514542 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.514516    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-home\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:45:06.514690 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.514552    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-tmp-dir\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:45:06.516231 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.516208    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-dshm" (OuterVolumeSpecName: "dshm") pod "71b30ffe-0950-4e90-ba03-cf43318fce22" (UID: "71b30ffe-0950-4e90-ba03-cf43318fce22"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:45:06.516231 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.516215    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71b30ffe-0950-4e90-ba03-cf43318fce22-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "71b30ffe-0950-4e90-ba03-cf43318fce22" (UID: "71b30ffe-0950-4e90-ba03-cf43318fce22"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:45:06.516390 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.516305    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71b30ffe-0950-4e90-ba03-cf43318fce22-kube-api-access-prtff" (OuterVolumeSpecName: "kube-api-access-prtff") pod "71b30ffe-0950-4e90-ba03-cf43318fce22" (UID: "71b30ffe-0950-4e90-ba03-cf43318fce22"). InnerVolumeSpecName "kube-api-access-prtff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:45:06.615537 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.615502    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-dshm\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:45:06.615537 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.615532    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prtff\" (UniqueName: \"kubernetes.io/projected/71b30ffe-0950-4e90-ba03-cf43318fce22-kube-api-access-prtff\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:45:06.615537 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.615544    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71b30ffe-0950-4e90-ba03-cf43318fce22-model-cache\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:45:06.615796 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.615552    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71b30ffe-0950-4e90-ba03-cf43318fce22-tls-certs\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:45:06.982726 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.982693    2575 generic.go:356] "Generic (PLEG): container finished" podID="71b30ffe-0950-4e90-ba03-cf43318fce22" containerID="8c489b22a9315c19edc3cdcbc3573066522c0d0723d3b5cc042940fdb7a8ce3c" exitCode=0
Jul 15 17:45:06.982921 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.982749    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"
Jul 15 17:45:06.982921 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.982751    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b" event={"ID":"71b30ffe-0950-4e90-ba03-cf43318fce22","Type":"ContainerDied","Data":"8c489b22a9315c19edc3cdcbc3573066522c0d0723d3b5cc042940fdb7a8ce3c"}
Jul 15 17:45:06.982921 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.982805    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b" event={"ID":"71b30ffe-0950-4e90-ba03-cf43318fce22","Type":"ContainerDied","Data":"d746ff582408e62793d4d0c8f2535b2508278aaff89b74314a542ec6abaa45d9"}
Jul 15 17:45:06.982921 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.982831    2575 scope.go:117] "RemoveContainer" containerID="8c489b22a9315c19edc3cdcbc3573066522c0d0723d3b5cc042940fdb7a8ce3c"
Jul 15 17:45:06.991392 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.991371    2575 scope.go:117] "RemoveContainer" containerID="8c489b22a9315c19edc3cdcbc3573066522c0d0723d3b5cc042940fdb7a8ce3c"
Jul 15 17:45:06.991667 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:45:06.991649    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c489b22a9315c19edc3cdcbc3573066522c0d0723d3b5cc042940fdb7a8ce3c\": container with ID starting with 8c489b22a9315c19edc3cdcbc3573066522c0d0723d3b5cc042940fdb7a8ce3c not found: ID does not exist" containerID="8c489b22a9315c19edc3cdcbc3573066522c0d0723d3b5cc042940fdb7a8ce3c"
Jul 15 17:45:06.991739 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:06.991679    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c489b22a9315c19edc3cdcbc3573066522c0d0723d3b5cc042940fdb7a8ce3c"} err="failed to get container status \"8c489b22a9315c19edc3cdcbc3573066522c0d0723d3b5cc042940fdb7a8ce3c\": rpc error: code = NotFound desc = could not find container \"8c489b22a9315c19edc3cdcbc3573066522c0d0723d3b5cc042940fdb7a8ce3c\": container with ID starting with 8c489b22a9315c19edc3cdcbc3573066522c0d0723d3b5cc042940fdb7a8ce3c not found: ID does not exist"
Jul 15 17:45:07.001787 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.001764    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"]
Jul 15 17:45:07.004502 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.004478    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-3d6c37/sub-test-model-3d6c37-kserve-66cb56c-qkt5b"]
Jul 15 17:45:07.105915 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.105871    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224" podUID="13188fb4-3507-4565-92a1-6cbf7868b499" containerName="main" containerID="cri-o://1f880f40e9a7581bb35055beae0a6d54052007df810321e37300c691f205e399" gracePeriod=45
Jul 15 17:45:07.353172 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.353143    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:45:07.423389 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.423345    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-tmp-dir\") pod \"13188fb4-3507-4565-92a1-6cbf7868b499\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") "
Jul 15 17:45:07.423592 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.423421    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13188fb4-3507-4565-92a1-6cbf7868b499-tls-certs\") pod \"13188fb4-3507-4565-92a1-6cbf7868b499\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") "
Jul 15 17:45:07.423592 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.423451    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-model-cache\") pod \"13188fb4-3507-4565-92a1-6cbf7868b499\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") "
Jul 15 17:45:07.423592 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.423503    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6lpp\" (UniqueName: \"kubernetes.io/projected/13188fb4-3507-4565-92a1-6cbf7868b499-kube-api-access-n6lpp\") pod \"13188fb4-3507-4565-92a1-6cbf7868b499\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") "
Jul 15 17:45:07.423844 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.423630    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-home\") pod \"13188fb4-3507-4565-92a1-6cbf7868b499\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") "
Jul 15 17:45:07.423844 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.423672    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "13188fb4-3507-4565-92a1-6cbf7868b499" (UID: "13188fb4-3507-4565-92a1-6cbf7868b499"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:45:07.423844 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.423718    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-dshm\") pod \"13188fb4-3507-4565-92a1-6cbf7868b499\" (UID: \"13188fb4-3507-4565-92a1-6cbf7868b499\") "
Jul 15 17:45:07.423844 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.423754    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-model-cache" (OuterVolumeSpecName: "model-cache") pod "13188fb4-3507-4565-92a1-6cbf7868b499" (UID: "13188fb4-3507-4565-92a1-6cbf7868b499"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:45:07.423996 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.423855    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-home" (OuterVolumeSpecName: "home") pod "13188fb4-3507-4565-92a1-6cbf7868b499" (UID: "13188fb4-3507-4565-92a1-6cbf7868b499"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:45:07.424082 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.424064    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-tmp-dir\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:45:07.424120 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.424089    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-model-cache\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:45:07.424120 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.424103    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-home\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:45:07.425649 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.425625    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13188fb4-3507-4565-92a1-6cbf7868b499-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "13188fb4-3507-4565-92a1-6cbf7868b499" (UID: "13188fb4-3507-4565-92a1-6cbf7868b499"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:45:07.425746 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.425677    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-dshm" (OuterVolumeSpecName: "dshm") pod "13188fb4-3507-4565-92a1-6cbf7868b499" (UID: "13188fb4-3507-4565-92a1-6cbf7868b499"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:45:07.425746 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.425697    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13188fb4-3507-4565-92a1-6cbf7868b499-kube-api-access-n6lpp" (OuterVolumeSpecName: "kube-api-access-n6lpp") pod "13188fb4-3507-4565-92a1-6cbf7868b499" (UID: "13188fb4-3507-4565-92a1-6cbf7868b499"). InnerVolumeSpecName "kube-api-access-n6lpp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:45:07.524684 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.524610    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13188fb4-3507-4565-92a1-6cbf7868b499-tls-certs\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:45:07.524684 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.524645    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6lpp\" (UniqueName: \"kubernetes.io/projected/13188fb4-3507-4565-92a1-6cbf7868b499-kube-api-access-n6lpp\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:45:07.524684 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.524654    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13188fb4-3507-4565-92a1-6cbf7868b499-dshm\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:45:07.988892 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.988853    2575 generic.go:356] "Generic (PLEG): container finished" podID="13188fb4-3507-4565-92a1-6cbf7868b499" containerID="1f880f40e9a7581bb35055beae0a6d54052007df810321e37300c691f205e399" exitCode=0
Jul 15 17:45:07.988892 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.988897    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"
Jul 15 17:45:07.989113 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.988934    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224" event={"ID":"13188fb4-3507-4565-92a1-6cbf7868b499","Type":"ContainerDied","Data":"1f880f40e9a7581bb35055beae0a6d54052007df810321e37300c691f205e399"}
Jul 15 17:45:07.989113 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.988978    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224" event={"ID":"13188fb4-3507-4565-92a1-6cbf7868b499","Type":"ContainerDied","Data":"30db0d8e024a563986e7b195a06f47214f631f0326538c8f18a5262604bf15ab"}
Jul 15 17:45:07.989113 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.988999    2575 scope.go:117] "RemoveContainer" containerID="1f880f40e9a7581bb35055beae0a6d54052007df810321e37300c691f205e399"
Jul 15 17:45:08.000014 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:07.999996    2575 scope.go:117] "RemoveContainer" containerID="1f880f40e9a7581bb35055beae0a6d54052007df810321e37300c691f205e399"
Jul 15 17:45:08.000241 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:45:08.000221    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f880f40e9a7581bb35055beae0a6d54052007df810321e37300c691f205e399\": container with ID starting with 1f880f40e9a7581bb35055beae0a6d54052007df810321e37300c691f205e399 not found: ID does not exist" containerID="1f880f40e9a7581bb35055beae0a6d54052007df810321e37300c691f205e399"
Jul 15 17:45:08.000310 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:08.000249    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f880f40e9a7581bb35055beae0a6d54052007df810321e37300c691f205e399"} err="failed to get container status \"1f880f40e9a7581bb35055beae0a6d54052007df810321e37300c691f205e399\": rpc error: code = NotFound desc = could not find container \"1f880f40e9a7581bb35055beae0a6d54052007df810321e37300c691f205e399\": container with ID starting with 1f880f40e9a7581bb35055beae0a6d54052007df810321e37300c691f205e399 not found: ID does not exist"
Jul 15 17:45:08.011908 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:08.011885    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"]
Jul 15 17:45:08.016014 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:08.015994    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-81a039/sub-test-model-81a039-kserve-67b8b4bff5-zr224"]
Jul 15 17:45:08.533156 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:08.533117    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13188fb4-3507-4565-92a1-6cbf7868b499" path="/var/lib/kubelet/pods/13188fb4-3507-4565-92a1-6cbf7868b499/volumes"
Jul 15 17:45:08.533519 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:45:08.533500    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71b30ffe-0950-4e90-ba03-cf43318fce22" path="/var/lib/kubelet/pods/71b30ffe-0950-4e90-ba03-cf43318fce22/volumes"
Jul 15 17:46:24.633611 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:24.633582    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 17:46:24.644402 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:24.644383    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kw2pl_9b91b45e-8528-42a0-ab1c-ac0b13c96c40/ovn-acl-logging/0.log"
Jul 15 17:46:43.451164 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:43.451124    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l"]
Jul 15 17:46:43.451601 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:43.451442    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l" podUID="b495ff27-5dff-43f5-b06c-f917cc7f75c7" containerName="maas-api" containerID="cri-o://4a287dad6a72bf823b8014852defefaad5ed5281b1c6448c32b3378792dea120" gracePeriod=30
Jul 15 17:46:43.712053 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:43.711998    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l"
Jul 15 17:46:43.812140 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:43.812109    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9bkw\" (UniqueName: \"kubernetes.io/projected/b495ff27-5dff-43f5-b06c-f917cc7f75c7-kube-api-access-p9bkw\") pod \"b495ff27-5dff-43f5-b06c-f917cc7f75c7\" (UID: \"b495ff27-5dff-43f5-b06c-f917cc7f75c7\") "
Jul 15 17:46:43.812300 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:43.812282    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b495ff27-5dff-43f5-b06c-f917cc7f75c7-openshift-service-ca\") pod \"b495ff27-5dff-43f5-b06c-f917cc7f75c7\" (UID: \"b495ff27-5dff-43f5-b06c-f917cc7f75c7\") "
Jul 15 17:46:43.812347 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:43.812301    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b495ff27-5dff-43f5-b06c-f917cc7f75c7-maas-api-tls\") pod \"b495ff27-5dff-43f5-b06c-f917cc7f75c7\" (UID: \"b495ff27-5dff-43f5-b06c-f917cc7f75c7\") "
Jul 15 17:46:43.812692 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:43.812665    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b495ff27-5dff-43f5-b06c-f917cc7f75c7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b495ff27-5dff-43f5-b06c-f917cc7f75c7" (UID: "b495ff27-5dff-43f5-b06c-f917cc7f75c7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:46:43.814029 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:43.814003    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b495ff27-5dff-43f5-b06c-f917cc7f75c7-kube-api-access-p9bkw" (OuterVolumeSpecName: "kube-api-access-p9bkw") pod "b495ff27-5dff-43f5-b06c-f917cc7f75c7" (UID: "b495ff27-5dff-43f5-b06c-f917cc7f75c7"). InnerVolumeSpecName "kube-api-access-p9bkw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:46:43.814225 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:43.814205    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b495ff27-5dff-43f5-b06c-f917cc7f75c7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b495ff27-5dff-43f5-b06c-f917cc7f75c7" (UID: "b495ff27-5dff-43f5-b06c-f917cc7f75c7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:46:43.913533 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:43.913495    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9bkw\" (UniqueName: \"kubernetes.io/projected/b495ff27-5dff-43f5-b06c-f917cc7f75c7-kube-api-access-p9bkw\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:46:43.913533 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:43.913524    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b495ff27-5dff-43f5-b06c-f917cc7f75c7-openshift-service-ca\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:46:43.913533 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:43.913534    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b495ff27-5dff-43f5-b06c-f917cc7f75c7-maas-api-tls\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:46:44.477441 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:44.477406    2575 generic.go:356] "Generic (PLEG): container finished" podID="b495ff27-5dff-43f5-b06c-f917cc7f75c7" containerID="4a287dad6a72bf823b8014852defefaad5ed5281b1c6448c32b3378792dea120" exitCode=0
Jul 15 17:46:44.477968 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:44.477450    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l"
Jul 15 17:46:44.477968 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:44.477475    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l" event={"ID":"b495ff27-5dff-43f5-b06c-f917cc7f75c7","Type":"ContainerDied","Data":"4a287dad6a72bf823b8014852defefaad5ed5281b1c6448c32b3378792dea120"}
Jul 15 17:46:44.477968 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:44.477522    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l" event={"ID":"b495ff27-5dff-43f5-b06c-f917cc7f75c7","Type":"ContainerDied","Data":"9e17cd543f52779fdaca01398785b45bf296e4c542b7344125e0f08da58ba2e6"}
Jul 15 17:46:44.477968 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:44.477544    2575 scope.go:117] "RemoveContainer" containerID="4a287dad6a72bf823b8014852defefaad5ed5281b1c6448c32b3378792dea120"
Jul 15 17:46:44.486724 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:44.486706    2575 scope.go:117] "RemoveContainer" containerID="4a287dad6a72bf823b8014852defefaad5ed5281b1c6448c32b3378792dea120"
Jul 15 17:46:44.486966 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:46:44.486948    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a287dad6a72bf823b8014852defefaad5ed5281b1c6448c32b3378792dea120\": container with ID starting with 4a287dad6a72bf823b8014852defefaad5ed5281b1c6448c32b3378792dea120 not found: ID does not exist" containerID="4a287dad6a72bf823b8014852defefaad5ed5281b1c6448c32b3378792dea120"
Jul 15 17:46:44.487016 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:44.486974    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a287dad6a72bf823b8014852defefaad5ed5281b1c6448c32b3378792dea120"} err="failed to get container status \"4a287dad6a72bf823b8014852defefaad5ed5281b1c6448c32b3378792dea120\": rpc error: code = NotFound desc = could not find container \"4a287dad6a72bf823b8014852defefaad5ed5281b1c6448c32b3378792dea120\": container with ID starting with 4a287dad6a72bf823b8014852defefaad5ed5281b1c6448c32b3378792dea120 not found: ID does not exist"
Jul 15 17:46:44.501418 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:44.501397    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l"]
Jul 15 17:46:44.505665 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:44.505639    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3d6c37-794c497594-mbz9l"]
Jul 15 17:46:44.526021 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:46:44.525995    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b495ff27-5dff-43f5-b06c-f917cc7f75c7" path="/var/lib/kubelet/pods/b495ff27-5dff-43f5-b06c-f917cc7f75c7/volumes"
Jul 15 17:47:09.689096 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:09.689061    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz"]
Jul 15 17:47:09.689617 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:09.689327    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz" podUID="3b4b29d3-597a-4bf1-b5f0-4a6e15181f29" containerName="maas-api" containerID="cri-o://5beb06acfa5a12c4f087d3eb99aea100a49a2437fdccdc57d14281a8933a80d5" gracePeriod=30
Jul 15 17:47:09.937636 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:09.937611    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz"
Jul 15 17:47:09.952474 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:09.952413    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b4b29d3-597a-4bf1-b5f0-4a6e15181f29-openshift-service-ca\") pod \"3b4b29d3-597a-4bf1-b5f0-4a6e15181f29\" (UID: \"3b4b29d3-597a-4bf1-b5f0-4a6e15181f29\") "
Jul 15 17:47:09.952474 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:09.952446    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b4b29d3-597a-4bf1-b5f0-4a6e15181f29-maas-api-tls\") pod \"3b4b29d3-597a-4bf1-b5f0-4a6e15181f29\" (UID: \"3b4b29d3-597a-4bf1-b5f0-4a6e15181f29\") "
Jul 15 17:47:09.952678 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:09.952505    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cz64f\" (UniqueName: \"kubernetes.io/projected/3b4b29d3-597a-4bf1-b5f0-4a6e15181f29-kube-api-access-cz64f\") pod \"3b4b29d3-597a-4bf1-b5f0-4a6e15181f29\" (UID: \"3b4b29d3-597a-4bf1-b5f0-4a6e15181f29\") "
Jul 15 17:47:09.952835 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:09.952807    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b4b29d3-597a-4bf1-b5f0-4a6e15181f29-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3b4b29d3-597a-4bf1-b5f0-4a6e15181f29" (UID: "3b4b29d3-597a-4bf1-b5f0-4a6e15181f29"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:47:09.954393 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:09.954371    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b4b29d3-597a-4bf1-b5f0-4a6e15181f29-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3b4b29d3-597a-4bf1-b5f0-4a6e15181f29" (UID: "3b4b29d3-597a-4bf1-b5f0-4a6e15181f29"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:47:09.954486 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:09.954432    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b4b29d3-597a-4bf1-b5f0-4a6e15181f29-kube-api-access-cz64f" (OuterVolumeSpecName: "kube-api-access-cz64f") pod "3b4b29d3-597a-4bf1-b5f0-4a6e15181f29" (UID: "3b4b29d3-597a-4bf1-b5f0-4a6e15181f29"). InnerVolumeSpecName "kube-api-access-cz64f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:47:10.053429 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:10.053393    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b4b29d3-597a-4bf1-b5f0-4a6e15181f29-openshift-service-ca\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:47:10.053429 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:10.053424    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b4b29d3-597a-4bf1-b5f0-4a6e15181f29-maas-api-tls\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:47:10.053429 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:10.053433    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cz64f\" (UniqueName: \"kubernetes.io/projected/3b4b29d3-597a-4bf1-b5f0-4a6e15181f29-kube-api-access-cz64f\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:47:10.614409 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:10.614377    2575 generic.go:356] "Generic (PLEG): container finished" podID="3b4b29d3-597a-4bf1-b5f0-4a6e15181f29" containerID="5beb06acfa5a12c4f087d3eb99aea100a49a2437fdccdc57d14281a8933a80d5" exitCode=0
Jul 15 17:47:10.614595 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:10.614420    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz"
Jul 15 17:47:10.614595 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:10.614416    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz" event={"ID":"3b4b29d3-597a-4bf1-b5f0-4a6e15181f29","Type":"ContainerDied","Data":"5beb06acfa5a12c4f087d3eb99aea100a49a2437fdccdc57d14281a8933a80d5"}
Jul 15 17:47:10.614595 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:10.614527    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz" event={"ID":"3b4b29d3-597a-4bf1-b5f0-4a6e15181f29","Type":"ContainerDied","Data":"524d80249d0c5e522a244554713b5cb04dfc45f9ccf5fe315b7d4031c661f5e6"}
Jul 15 17:47:10.614595 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:10.614552    2575 scope.go:117] "RemoveContainer" containerID="5beb06acfa5a12c4f087d3eb99aea100a49a2437fdccdc57d14281a8933a80d5"
Jul 15 17:47:10.623674 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:10.623657    2575 scope.go:117] "RemoveContainer" containerID="5beb06acfa5a12c4f087d3eb99aea100a49a2437fdccdc57d14281a8933a80d5"
Jul 15 17:47:10.623945 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:47:10.623919    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5beb06acfa5a12c4f087d3eb99aea100a49a2437fdccdc57d14281a8933a80d5\": container with ID starting with 5beb06acfa5a12c4f087d3eb99aea100a49a2437fdccdc57d14281a8933a80d5 not found: ID does not exist" containerID="5beb06acfa5a12c4f087d3eb99aea100a49a2437fdccdc57d14281a8933a80d5"
Jul 15 17:47:10.624005 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:10.623945    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5beb06acfa5a12c4f087d3eb99aea100a49a2437fdccdc57d14281a8933a80d5"} err="failed to get container status \"5beb06acfa5a12c4f087d3eb99aea100a49a2437fdccdc57d14281a8933a80d5\": rpc error: code = NotFound desc = could not find container \"5beb06acfa5a12c4f087d3eb99aea100a49a2437fdccdc57d14281a8933a80d5\": container with ID starting with 5beb06acfa5a12c4f087d3eb99aea100a49a2437fdccdc57d14281a8933a80d5 not found: ID does not exist"
Jul 15 17:47:10.631315 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:10.631294    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz"]
Jul 15 17:47:10.635687 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:10.635666    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-81a039-65f4b9fccc-gplhz"]
Jul 15 17:47:12.525927 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:12.525889    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b4b29d3-597a-4bf1-b5f0-4a6e15181f29" path="/var/lib/kubelet/pods/3b4b29d3-597a-4bf1-b5f0-4a6e15181f29/volumes"
Jul 15 17:47:24.774130 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:24.774090    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"]
Jul 15 17:47:24.774612 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:24.774356    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2" podUID="0149e98f-7f35-4f1c-b29c-f2bbab3f8006" containerName="istio-proxy" containerID="cri-o://25dac47eba6c2dec07444e653ce4be5973d2a24baff5bb41c21a6acdbe600b8e" gracePeriod=30
Jul 15 17:47:30.033922 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.033898    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:47:30.124705 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.124680    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5z55\" (UniqueName: \"kubernetes.io/projected/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-kube-api-access-t5z55\") pod \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") "
Jul 15 17:47:30.124866 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.124730    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-token\") pod \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") "
Jul 15 17:47:30.124866 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.124775    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-podinfo\") pod \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") "
Jul 15 17:47:30.124866 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.124802    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-workload-socket\") pod \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") "
Jul 15 17:47:30.124866 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.124835    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-credential-socket\") pod \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") "
Jul 15 17:47:30.125072 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.124880    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-envoy\") pod \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") "
Jul 15 17:47:30.125072 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.124909    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istiod-ca-cert\") pod \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") "
Jul 15 17:47:30.125072 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.124944    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-data\") pod \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") "
Jul 15 17:47:30.125072 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.125002    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-workload-certs\") pod \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\" (UID: \"0149e98f-7f35-4f1c-b29c-f2bbab3f8006\") "
Jul 15 17:47:30.125271 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.125179    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0149e98f-7f35-4f1c-b29c-f2bbab3f8006" (UID: "0149e98f-7f35-4f1c-b29c-f2bbab3f8006"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:47:30.125271 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.125195    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0149e98f-7f35-4f1c-b29c-f2bbab3f8006" (UID: "0149e98f-7f35-4f1c-b29c-f2bbab3f8006"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:47:30.125409 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.125384    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-workload-socket\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:47:30.125463 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.125410    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-credential-socket\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:47:30.125463 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.125388    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0149e98f-7f35-4f1c-b29c-f2bbab3f8006" (UID: "0149e98f-7f35-4f1c-b29c-f2bbab3f8006"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:47:30.126082 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.126053    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0149e98f-7f35-4f1c-b29c-f2bbab3f8006" (UID: "0149e98f-7f35-4f1c-b29c-f2bbab3f8006"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:47:30.126207 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.126190    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-data" (OuterVolumeSpecName: "istio-data") pod "0149e98f-7f35-4f1c-b29c-f2bbab3f8006" (UID: "0149e98f-7f35-4f1c-b29c-f2bbab3f8006"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:47:30.127074 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.127049    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-kube-api-access-t5z55" (OuterVolumeSpecName: "kube-api-access-t5z55") pod "0149e98f-7f35-4f1c-b29c-f2bbab3f8006" (UID: "0149e98f-7f35-4f1c-b29c-f2bbab3f8006"). InnerVolumeSpecName "kube-api-access-t5z55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:47:30.127185 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.127083    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0149e98f-7f35-4f1c-b29c-f2bbab3f8006" (UID: "0149e98f-7f35-4f1c-b29c-f2bbab3f8006"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:47:30.127185 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.127124    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-token" (OuterVolumeSpecName: "istio-token") pod "0149e98f-7f35-4f1c-b29c-f2bbab3f8006" (UID: "0149e98f-7f35-4f1c-b29c-f2bbab3f8006"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:47:30.127350 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.127328    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0149e98f-7f35-4f1c-b29c-f2bbab3f8006" (UID: "0149e98f-7f35-4f1c-b29c-f2bbab3f8006"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 17:47:30.226211 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.226178    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5z55\" (UniqueName: \"kubernetes.io/projected/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-kube-api-access-t5z55\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:47:30.226211 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.226205    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-token\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:47:30.226211 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.226215    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-podinfo\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:47:30.226434 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.226223    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-envoy\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:47:30.226434 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.226230    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istiod-ca-cert\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:47:30.226434 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.226237    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-istio-data\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:47:30.226434 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.226245    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0149e98f-7f35-4f1c-b29c-f2bbab3f8006-workload-certs\") on node \"ip-10-0-142-51.ec2.internal\" DevicePath \"\""
Jul 15 17:47:30.715190 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.715155    2575 generic.go:356] "Generic (PLEG): container finished" podID="0149e98f-7f35-4f1c-b29c-f2bbab3f8006" containerID="25dac47eba6c2dec07444e653ce4be5973d2a24baff5bb41c21a6acdbe600b8e" exitCode=0
Jul 15 17:47:30.715391 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.715213    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"
Jul 15 17:47:30.715391 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.715231    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2" event={"ID":"0149e98f-7f35-4f1c-b29c-f2bbab3f8006","Type":"ContainerDied","Data":"25dac47eba6c2dec07444e653ce4be5973d2a24baff5bb41c21a6acdbe600b8e"}
Jul 15 17:47:30.715391 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.715281    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2" event={"ID":"0149e98f-7f35-4f1c-b29c-f2bbab3f8006","Type":"ContainerDied","Data":"b2bbe88fea00358851258a8a098c1a4fbaf70a3edd1ee671479faa8f3c22e1d9"}
Jul 15 17:47:30.715391 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.715303    2575 scope.go:117] "RemoveContainer" containerID="25dac47eba6c2dec07444e653ce4be5973d2a24baff5bb41c21a6acdbe600b8e"
Jul 15 17:47:30.723848 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.723834    2575 scope.go:117] "RemoveContainer" containerID="61b942c27da89ab5a8d12773629e560853715af1fbf905d79a76893c0adc2bd0"
Jul 15 17:47:30.733895 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.733877    2575 scope.go:117] "RemoveContainer" containerID="25dac47eba6c2dec07444e653ce4be5973d2a24baff5bb41c21a6acdbe600b8e"
Jul 15 17:47:30.734200 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:47:30.734179    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25dac47eba6c2dec07444e653ce4be5973d2a24baff5bb41c21a6acdbe600b8e\": container with ID starting with 25dac47eba6c2dec07444e653ce4be5973d2a24baff5bb41c21a6acdbe600b8e not found: ID does not exist" containerID="25dac47eba6c2dec07444e653ce4be5973d2a24baff5bb41c21a6acdbe600b8e"
Jul 15 17:47:30.734268 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.734209    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25dac47eba6c2dec07444e653ce4be5973d2a24baff5bb41c21a6acdbe600b8e"} err="failed to get container status \"25dac47eba6c2dec07444e653ce4be5973d2a24baff5bb41c21a6acdbe600b8e\": rpc error: code = NotFound desc = could not find container \"25dac47eba6c2dec07444e653ce4be5973d2a24baff5bb41c21a6acdbe600b8e\": container with ID starting with 25dac47eba6c2dec07444e653ce4be5973d2a24baff5bb41c21a6acdbe600b8e not found: ID does not exist"
Jul 15 17:47:30.734268 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.734225    2575 scope.go:117] "RemoveContainer" containerID="61b942c27da89ab5a8d12773629e560853715af1fbf905d79a76893c0adc2bd0"
Jul 15 17:47:30.734412 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.734393    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"]
Jul 15 17:47:30.734454 ip-10-0-142-51 kubenswrapper[2575]: E0715 17:47:30.734439    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61b942c27da89ab5a8d12773629e560853715af1fbf905d79a76893c0adc2bd0\": container with ID starting with 61b942c27da89ab5a8d12773629e560853715af1fbf905d79a76893c0adc2bd0 not found: ID does not exist" containerID="61b942c27da89ab5a8d12773629e560853715af1fbf905d79a76893c0adc2bd0"
Jul 15 17:47:30.734490 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.734459    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61b942c27da89ab5a8d12773629e560853715af1fbf905d79a76893c0adc2bd0"} err="failed to get container status \"61b942c27da89ab5a8d12773629e560853715af1fbf905d79a76893c0adc2bd0\": rpc error: code = NotFound desc = could not find container \"61b942c27da89ab5a8d12773629e560853715af1fbf905d79a76893c0adc2bd0\": container with ID starting with 61b942c27da89ab5a8d12773629e560853715af1fbf905d79a76893c0adc2bd0 not found: ID does not exist"
Jul 15 17:47:30.738381 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:30.738360    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-81a039-openshift-default-55dcd5b7-b6lv2"]
Jul 15 17:47:31.004056 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:31.004029    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4hfx2_2cc8746d-b197-452a-98d2-e89febe73f6c/manager/0.log"
Jul 15 17:47:31.135700 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:31.135673    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-f8zrm_f996b798-29d1-4cca-85fe-e8394d89ce9b/manager/0.log"
Jul 15 17:47:31.260470 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:31.260394    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6b8d7f4fcb-84rmq_39851bee-faca-4dea-843c-4baa4695efc3/manager/0.log"
Jul 15 17:47:31.375572 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:31.375529    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-frrfv_191790f8-4646-4b5f-af3e-667b4fbf9af2/server/0.log"
Jul 15 17:47:31.500956 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:31.500933    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-8zhqd_47c4c998-668f-4026-9f1d-95f3a398b7f9/manager/0.log"
Jul 15 17:47:31.903793 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:31.903765    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b767d9f5c-q4zv5_df8d2fcf-ace8-4e7e-a47a-0f5342b1411d/manager/0.log"
Jul 15 17:47:31.911030 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:31.911006    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b767d9f5c-q4zv5_df8d2fcf-ace8-4e7e-a47a-0f5342b1411d/copy-manifests/0.log"
Jul 15 17:47:32.294138 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:32.294041    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7f869c5dd8-t4f6b_89b150b3-cdf8-4bc2-80db-03a6c7748aaa/maas-api/0.log"
Jul 15 17:47:32.525440 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:32.525406    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0149e98f-7f35-4f1c-b29c-f2bbab3f8006" path="/var/lib/kubelet/pods/0149e98f-7f35-4f1c-b29c-f2bbab3f8006/volumes"
Jul 15 17:47:34.148872 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:34.148844    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-2zrsn_e35db717-b98d-4336-b4db-e1a411902f2f/postgres/0.log"
Jul 15 17:47:35.626121 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:35.626094    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-z6dxh_54f5f063-919a-4b65-b250-feb767ff295d/manager/0.log"
Jul 15 17:47:35.746512 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:35.746479    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-f7dfk_376e83c4-799d-4cb4-9d9c-062c949edc8e/manager/0.log"
Jul 15 17:47:35.859390 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:35.859356    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-nmtff_b6facd58-286a-47fc-9226-5ccdb6d8113e/kuadrant-console-plugin/0.log"
Jul 15 17:47:35.978922 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:35.978845    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-bpn7c_c111ce03-5c9b-4c21-b67d-89a7c081a198/registry-server/0.log"
Jul 15 17:47:36.348160 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:36.348087    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-w9p25_7d57e791-c22c-424f-9612-1960f235de30/manager/0.log"
Jul 15 17:47:36.704853 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:36.704820    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4xd8ck_7ea1850f-81f0-4321-be0d-fcbe74517f5d/istio-proxy/0.log"
Jul 15 17:47:37.065852 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:37.065762    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b44776b5f-m2gtp_7beab3a5-bfb9-435a-93c5-3ba8265d6ad0/kube-auth-proxy/0.log"
Jul 15 17:47:37.182657 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:37.182628    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-nhsnr_01190a7c-1f78-4e65-8582-73df15d76a5e/istio-proxy/1.log"
Jul 15 17:47:37.302078 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:37.302048    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-7f7qf_89a262f5-ce88-4325-9cca-ab0eaf765cac/payload-pre-processing/0.log"
Jul 15 17:47:37.422446 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:37.422422    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-ngw49_353226b7-3a58-48c2-9f48-610b86e44f37/payload-processing/0.log"
Jul 15 17:47:37.891045 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:37.891014    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-6bwtm_573d4a56-b7ee-4f30-97e7-1f985ead79b4/main/0.log"
Jul 15 17:47:38.126317 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:38.126287    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-mbl76_e9933c89-3226-4ce2-9900-675eaeccc0ed/main/0.log"
Jul 15 17:47:38.473427 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:38.473400    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-jdhgw_d90e633a-075d-4944-958f-af94b19ffee4/main/0.log"
Jul 15 17:47:46.878530 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:46.878498    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bnz8n_ae13fea8-2c11-48da-b201-4007b9b6272c/global-pull-secret-syncer/0.log"
Jul 15 17:47:47.046390 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:47.046361    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vf8th_73f51511-7a5f-46dc-b252-9d05fdcefb15/konnectivity-agent/0.log"
Jul 15 17:47:47.118067 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:47.118036    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-51.ec2.internal_ef7caf07f6dae51e575ab6538fd06a86/haproxy/0.log"
Jul 15 17:47:52.820910 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:52.820879    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-z6dxh_54f5f063-919a-4b65-b250-feb767ff295d/manager/0.log"
Jul 15 17:47:52.854246 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:52.854218    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-f7dfk_376e83c4-799d-4cb4-9d9c-062c949edc8e/manager/0.log"
Jul 15 17:47:52.882609 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:52.882587    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-nmtff_b6facd58-286a-47fc-9226-5ccdb6d8113e/kuadrant-console-plugin/0.log"
Jul 15 17:47:52.926796 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:52.926772    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-bpn7c_c111ce03-5c9b-4c21-b67d-89a7c081a198/registry-server/0.log"
Jul 15 17:47:53.242823 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:53.242794    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-w9p25_7d57e791-c22c-424f-9612-1960f235de30/manager/0.log"
Jul 15 17:47:55.285256 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:55.285227    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8d7c5_b74b3ae7-7e36-4419-be70-b15a30fe8cf9/node-exporter/0.log"
Jul 15 17:47:55.309417 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:55.309395    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8d7c5_b74b3ae7-7e36-4419-be70-b15a30fe8cf9/kube-rbac-proxy/0.log"
Jul 15 17:47:55.332030 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:55.332008    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8d7c5_b74b3ae7-7e36-4419-be70-b15a30fe8cf9/init-textfile/0.log"
Jul 15 17:47:55.789144 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:55.789116    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-6w956_e3efe260-102a-456d-9c7d-a47277cf1f35/prometheus-operator-admission-webhook/0.log"
Jul 15 17:47:56.600842 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.600809    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w"]
Jul 15 17:47:56.601209 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.601146    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13188fb4-3507-4565-92a1-6cbf7868b499" containerName="main"
Jul 15 17:47:56.601209 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.601156    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="13188fb4-3507-4565-92a1-6cbf7868b499" containerName="main"
Jul 15 17:47:56.601209 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.601165    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0149e98f-7f35-4f1c-b29c-f2bbab3f8006" containerName="istio-proxy"
Jul 15 17:47:56.601209 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.601172    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0149e98f-7f35-4f1c-b29c-f2bbab3f8006" containerName="istio-proxy"
Jul 15 17:47:56.601209 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.601189    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71b30ffe-0950-4e90-ba03-cf43318fce22" containerName="main"
Jul 15 17:47:56.601209 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.601195    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="71b30ffe-0950-4e90-ba03-cf43318fce22" containerName="main"
Jul 15 17:47:56.601209 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.601209    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b4b29d3-597a-4bf1-b5f0-4a6e15181f29" containerName="maas-api"
Jul 15 17:47:56.601209 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.601214    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b4b29d3-597a-4bf1-b5f0-4a6e15181f29" containerName="maas-api"
Jul 15 17:47:56.601461 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.601221    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b495ff27-5dff-43f5-b06c-f917cc7f75c7" containerName="maas-api"
Jul 15 17:47:56.601461 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.601226    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b495ff27-5dff-43f5-b06c-f917cc7f75c7" containerName="maas-api"
Jul 15 17:47:56.601461 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.601277    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b4b29d3-597a-4bf1-b5f0-4a6e15181f29" containerName="maas-api"
Jul 15 17:47:56.601461 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.601284    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0149e98f-7f35-4f1c-b29c-f2bbab3f8006" containerName="istio-proxy"
Jul 15 17:47:56.601461 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.601290    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b495ff27-5dff-43f5-b06c-f917cc7f75c7" containerName="maas-api"
Jul 15 17:47:56.601461 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.601298    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="13188fb4-3507-4565-92a1-6cbf7868b499" containerName="main"
Jul 15 17:47:56.601461 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.601306    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="71b30ffe-0950-4e90-ba03-cf43318fce22" containerName="main"
Jul 15 17:47:56.605467 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.605445    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w"
Jul 15 17:47:56.614543 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.614519    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8cbcq\"/\"openshift-service-ca.crt\""
Jul 15 17:47:56.614655 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.614547    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8cbcq\"/\"kube-root-ca.crt\""
Jul 15 17:47:56.615504 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.615479    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8cbcq\"/\"default-dockercfg-zrwf4\""
Jul 15 17:47:56.617780 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.617763    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w"]
Jul 15 17:47:56.659480 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.659452    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eee74428-9344-4fae-b16e-025577e23687-proc\") pod \"perf-node-gather-daemonset-7sm9w\" (UID: \"eee74428-9344-4fae-b16e-025577e23687\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w"
Jul 15 17:47:56.659624 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.659488    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl2qk\" (UniqueName: \"kubernetes.io/projected/eee74428-9344-4fae-b16e-025577e23687-kube-api-access-cl2qk\") pod \"perf-node-gather-daemonset-7sm9w\" (UID: \"eee74428-9344-4fae-b16e-025577e23687\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w"
Jul 15 17:47:56.659624 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.659578    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eee74428-9344-4fae-b16e-025577e23687-sys\") pod \"perf-node-gather-daemonset-7sm9w\" (UID: \"eee74428-9344-4fae-b16e-025577e23687\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w"
Jul 15 17:47:56.659711 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.659651    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eee74428-9344-4fae-b16e-025577e23687-podres\") pod \"perf-node-gather-daemonset-7sm9w\" (UID: \"eee74428-9344-4fae-b16e-025577e23687\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w"
Jul 15 17:47:56.659711 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.659678    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eee74428-9344-4fae-b16e-025577e23687-lib-modules\") pod \"perf-node-gather-daemonset-7sm9w\" (UID: \"eee74428-9344-4fae-b16e-025577e23687\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w"
Jul 15 17:47:56.760770 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.760742    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eee74428-9344-4fae-b16e-025577e23687-sys\") pod \"perf-node-gather-daemonset-7sm9w\" (UID: \"eee74428-9344-4fae-b16e-025577e23687\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w"
Jul 15 17:47:56.760917 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.760791    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eee74428-9344-4fae-b16e-025577e23687-podres\") pod \"perf-node-gather-daemonset-7sm9w\" (UID: \"eee74428-9344-4fae-b16e-025577e23687\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w"
Jul 15 17:47:56.760917 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.760813    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eee74428-9344-4fae-b16e-025577e23687-lib-modules\") pod \"perf-node-gather-daemonset-7sm9w\" (UID: \"eee74428-9344-4fae-b16e-025577e23687\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w"
Jul 15 17:47:56.760917 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.760849    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eee74428-9344-4fae-b16e-025577e23687-proc\") pod \"perf-node-gather-daemonset-7sm9w\" (UID: \"eee74428-9344-4fae-b16e-025577e23687\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w"
Jul 15 17:47:56.760917 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.760874    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl2qk\" (UniqueName: \"kubernetes.io/projected/eee74428-9344-4fae-b16e-025577e23687-kube-api-access-cl2qk\") pod \"perf-node-gather-daemonset-7sm9w\" (UID: \"eee74428-9344-4fae-b16e-025577e23687\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w"
Jul 15 17:47:56.760917 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.760849    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eee74428-9344-4fae-b16e-025577e23687-sys\") pod \"perf-node-gather-daemonset-7sm9w\" (UID: \"eee74428-9344-4fae-b16e-025577e23687\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w"
Jul 15 17:47:56.761125 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.760931    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eee74428-9344-4fae-b16e-025577e23687-proc\") pod \"perf-node-gather-daemonset-7sm9w\" (UID: \"eee74428-9344-4fae-b16e-025577e23687\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w"
Jul 15 17:47:56.761125 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.760932    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eee74428-9344-4fae-b16e-025577e23687-podres\") pod \"perf-node-gather-daemonset-7sm9w\" (UID: \"eee74428-9344-4fae-b16e-025577e23687\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w"
Jul 15 17:47:56.761125 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.760965    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eee74428-9344-4fae-b16e-025577e23687-lib-modules\") pod \"perf-node-gather-daemonset-7sm9w\" (UID: \"eee74428-9344-4fae-b16e-025577e23687\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w"
Jul 15 17:47:56.770364 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.770336    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl2qk\" (UniqueName: \"kubernetes.io/projected/eee74428-9344-4fae-b16e-025577e23687-kube-api-access-cl2qk\") pod \"perf-node-gather-daemonset-7sm9w\" (UID: \"eee74428-9344-4fae-b16e-025577e23687\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w"
Jul 15 17:47:56.914750 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:56.914727    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w"
Jul 15 17:47:57.044474 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:57.044438    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w"]
Jul 15 17:47:57.046301 ip-10-0-142-51 kubenswrapper[2575]: W0715 17:47:57.046276    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podeee74428_9344_4fae_b16e_025577e23687.slice/crio-7ffc8b797bcc5eb2d5de09e8b6d04aaaab224843fd92ae4c1bd7913da15535b7 WatchSource:0}: Error finding container 7ffc8b797bcc5eb2d5de09e8b6d04aaaab224843fd92ae4c1bd7913da15535b7: Status 404 returned error can't find the container with id 7ffc8b797bcc5eb2d5de09e8b6d04aaaab224843fd92ae4c1bd7913da15535b7
Jul 15 17:47:57.843415 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:57.843377    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w" event={"ID":"eee74428-9344-4fae-b16e-025577e23687","Type":"ContainerStarted","Data":"fcd82c93ae673820c04e307085653e5af96b185aab07346f0edf47af6b19c053"}
Jul 15 17:47:57.843865 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:57.843431    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w" event={"ID":"eee74428-9344-4fae-b16e-025577e23687","Type":"ContainerStarted","Data":"7ffc8b797bcc5eb2d5de09e8b6d04aaaab224843fd92ae4c1bd7913da15535b7"}
Jul 15 17:47:57.843865 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:57.843452    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w"
Jul 15 17:47:57.862048 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:57.862000    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w" podStartSLOduration=1.86198454 podStartE2EDuration="1.86198454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 17:47:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:47:57.860390911 +0000 UTC m=+3093.877804025" watchObservedRunningTime="2026-07-15 17:47:57.86198454 +0000 UTC m=+3093.879397636"
Jul 15 17:47:58.198161 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:58.198132    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7f6cd4fd5c-csqmm_fe433fb2-8705-4c66-bc23-4cae16489508/console/0.log"
Jul 15 17:47:59.893142 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:59.893117    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f6q75_2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3/dns/0.log"
Jul 15 17:47:59.914530 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:47:59.914502    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f6q75_2cd5fea3-92ab-4a77-8b89-a91ef90aa2a3/kube-rbac-proxy/0.log"
Jul 15 17:48:00.008901 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:00.008873    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jdrr4_c1de2bfc-4272-4761-b27a-76d5c4f9949e/dns-node-resolver/0.log"
Jul 15 17:48:00.616730 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:00.616692    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2brrq_5d97b3f4-a52b-4bd8-bf1a-ab72f5304ea4/node-ca/0.log"
Jul 15 17:48:01.619861 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:01.619831    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4xd8ck_7ea1850f-81f0-4321-be0d-fcbe74517f5d/istio-proxy/0.log"
Jul 15 17:48:01.794484 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:01.794457    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b44776b5f-m2gtp_7beab3a5-bfb9-435a-93c5-3ba8265d6ad0/kube-auth-proxy/0.log"
Jul 15 17:48:01.823257 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:01.823227    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-nhsnr_01190a7c-1f78-4e65-8582-73df15d76a5e/istio-proxy/0.log"
Jul 15 17:48:01.849861 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:01.849829    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-nhsnr_01190a7c-1f78-4e65-8582-73df15d76a5e/istio-proxy/1.log"
Jul 15 17:48:01.878086 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:01.878009    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-7f7qf_89a262f5-ce88-4325-9cca-ab0eaf765cac/payload-pre-processing/0.log"
Jul 15 17:48:01.905772 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:01.905746    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-ngw49_353226b7-3a58-48c2-9f48-610b86e44f37/payload-processing/0.log"
Jul 15 17:48:02.695990 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:02.695959    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p2qjz_00f13bdd-a991-489f-879a-019b72447eb5/serve-healthcheck-canary/0.log"
Jul 15 17:48:03.224730 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:03.224704    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tlgvp_fee99ffe-3358-4efa-8014-a66f6a973724/kube-rbac-proxy/0.log"
Jul 15 17:48:03.243992 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:03.243967    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tlgvp_fee99ffe-3358-4efa-8014-a66f6a973724/exporter/0.log"
Jul 15 17:48:03.263540 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:03.263520    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tlgvp_fee99ffe-3358-4efa-8014-a66f6a973724/extractor/0.log"
Jul 15 17:48:03.861862 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:03.861824    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-7sm9w"
Jul 15 17:48:05.544368 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:05.544341    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4hfx2_2cc8746d-b197-452a-98d2-e89febe73f6c/manager/0.log"
Jul 15 17:48:05.684711 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:05.684686    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-f8zrm_f996b798-29d1-4cca-85fe-e8394d89ce9b/manager/0.log"
Jul 15 17:48:05.784850 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:05.784823    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6b8d7f4fcb-84rmq_39851bee-faca-4dea-843c-4baa4695efc3/manager/0.log"
Jul 15 17:48:05.809371 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:05.809313    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-frrfv_191790f8-4646-4b5f-af3e-667b4fbf9af2/server/0.log"
Jul 15 17:48:05.879427 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:05.879402    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-8zhqd_47c4c998-668f-4026-9f1d-95f3a398b7f9/manager/0.log"
Jul 15 17:48:06.107546 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:06.107470    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b767d9f5c-q4zv5_df8d2fcf-ace8-4e7e-a47a-0f5342b1411d/manager/0.log"
Jul 15 17:48:06.126540 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:06.126518    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b767d9f5c-q4zv5_df8d2fcf-ace8-4e7e-a47a-0f5342b1411d/copy-manifests/0.log"
Jul 15 17:48:07.331156 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:07.331081    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4xd8ck_7ea1850f-81f0-4321-be0d-fcbe74517f5d/istio-proxy/0.log"
Jul 15 17:48:07.467514 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:07.467484    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b44776b5f-m2gtp_7beab3a5-bfb9-435a-93c5-3ba8265d6ad0/kube-auth-proxy/0.log"
Jul 15 17:48:07.492525 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:07.492503    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-nhsnr_01190a7c-1f78-4e65-8582-73df15d76a5e/istio-proxy/0.log"
Jul 15 17:48:07.523526 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:07.523507    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-nhsnr_01190a7c-1f78-4e65-8582-73df15d76a5e/istio-proxy/1.log"
Jul 15 17:48:07.546589 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:07.546567    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-7f7qf_89a262f5-ce88-4325-9cca-ab0eaf765cac/payload-pre-processing/0.log"
Jul 15 17:48:07.566116 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:07.566092    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-ngw49_353226b7-3a58-48c2-9f48-610b86e44f37/payload-processing/0.log"
Jul 15 17:48:07.625781 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:07.625764    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-67fcj_77bc1858-8821-4fb9-a70b-61956adaa46e/openshift-lws-operator/0.log"
Jul 15 17:48:08.718846 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:08.718825    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4xd8ck_7ea1850f-81f0-4321-be0d-fcbe74517f5d/istio-proxy/0.log"
Jul 15 17:48:08.870498 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:08.870472    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b44776b5f-m2gtp_7beab3a5-bfb9-435a-93c5-3ba8265d6ad0/kube-auth-proxy/0.log"
Jul 15 17:48:08.887878 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:08.887838    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-nhsnr_01190a7c-1f78-4e65-8582-73df15d76a5e/istio-proxy/0.log"
Jul 15 17:48:08.918026 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:08.918009    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-nhsnr_01190a7c-1f78-4e65-8582-73df15d76a5e/istio-proxy/1.log"
Jul 15 17:48:08.942149 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:08.942108    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-7f7qf_89a262f5-ce88-4325-9cca-ab0eaf765cac/payload-pre-processing/0.log"
Jul 15 17:48:08.963201 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:08.963173    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-ngw49_353226b7-3a58-48c2-9f48-610b86e44f37/payload-processing/0.log"
Jul 15 17:48:14.202570 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:14.202527    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mnvjv_adca27dc-3730-43de-9940-070f0bf616f9/kube-multus-additional-cni-plugins/0.log"
Jul 15 17:48:14.221233 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:14.221207    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mnvjv_adca27dc-3730-43de-9940-070f0bf616f9/egress-router-binary-copy/0.log"
Jul 15 17:48:14.241733 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:14.241710    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mnvjv_adca27dc-3730-43de-9940-070f0bf616f9/cni-plugins/0.log"
Jul 15 17:48:14.260736 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:14.260715    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mnvjv_adca27dc-3730-43de-9940-070f0bf616f9/bond-cni-plugin/0.log"
Jul 15 17:48:14.280834 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:14.280815    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mnvjv_adca27dc-3730-43de-9940-070f0bf616f9/routeoverride-cni/0.log"
Jul 15 17:48:14.299106 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:14.299085    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mnvjv_adca27dc-3730-43de-9940-070f0bf616f9/whereabouts-cni-bincopy/0.log"
Jul 15 17:48:14.318597 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:14.318573    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mnvjv_adca27dc-3730-43de-9940-070f0bf616f9/whereabouts-cni/0.log"
Jul 15 17:48:14.504096 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:14.504016    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fxndg_38685520-a6e8-4e97-bb11-63fcc732e3ba/kube-multus/0.log"
Jul 15 17:48:14.569297 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:14.569271    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7kv7g_a62f9005-2c66-4d30-9102-121aae529be8/network-metrics-daemon/0.log"
Jul 15 17:48:14.587770 ip-10-0-142-51 kubenswrapper[2575]: I0715 17:48:14.587748    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7kv7g_a62f9005-2c66-4d30-9102-121aae529be8/kube-rbac-proxy/0.log"