Jul 28 22:04:30.909550 ip-10-0-143-69 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 22:04:31.246138 ip-10-0-143-69 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 22:04:31.246138 ip-10-0-143-69 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 22:04:31.246138 ip-10-0-143-69 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 22:04:31.246138 ip-10-0-143-69 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 22:04:31.246138 ip-10-0-143-69 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 22:04:31.246920 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.246808    2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 22:04:31.251250 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251212    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 22:04:31.251250 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251248    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 22:04:31.251387 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251255    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 22:04:31.251387 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251266    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 22:04:31.251387 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251271    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 22:04:31.251387 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251275    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 22:04:31.251387 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251283    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 22:04:31.251387 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251289    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 22:04:31.251387 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251294    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 22:04:31.251387 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251299    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 22:04:31.251387 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251303    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 22:04:31.251387 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251308    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 22:04:31.251387 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251312    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 22:04:31.251387 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251316    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 22:04:31.251387 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251321    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 22:04:31.251387 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251333    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 22:04:31.251387 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251338    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 22:04:31.251387 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251343    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 22:04:31.251387 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251348    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 22:04:31.251387 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251352    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 22:04:31.251387 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251356    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 22:04:31.252077 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251361    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 22:04:31.252077 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251365    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 22:04:31.252077 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251372    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 22:04:31.252077 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251377    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 22:04:31.252077 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251382    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 22:04:31.252077 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251387    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 22:04:31.252077 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251397    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 22:04:31.252077 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251403    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 22:04:31.252077 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251407    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 22:04:31.252077 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251412    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 22:04:31.252077 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251416    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 22:04:31.252077 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251420    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 22:04:31.252077 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251424    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 22:04:31.252077 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251428    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 22:04:31.252077 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251432    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 22:04:31.252077 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251437    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 22:04:31.252077 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251441    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 22:04:31.252077 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251447    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 22:04:31.252077 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251456    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 22:04:31.252625 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251460    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 22:04:31.252625 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251464    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 22:04:31.252625 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251469    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 22:04:31.252625 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251473    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 22:04:31.252625 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251477    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 22:04:31.252625 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251481    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 22:04:31.252625 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251485    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 22:04:31.252625 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251489    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 22:04:31.252625 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251493    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 22:04:31.252625 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251497    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 22:04:31.252625 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251501    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 22:04:31.252625 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251505    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 22:04:31.252625 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251513    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 22:04:31.252625 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251517    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 22:04:31.252625 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251523    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 22:04:31.252625 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251528    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 22:04:31.252625 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251535    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 22:04:31.252625 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251539    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 22:04:31.252625 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251543    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 22:04:31.252625 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251548    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 22:04:31.253148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251553    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 22:04:31.253148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251557    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 22:04:31.253148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251561    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 22:04:31.253148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251571    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 22:04:31.253148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251575    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 22:04:31.253148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251579    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 22:04:31.253148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251584    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 22:04:31.253148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251588    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 22:04:31.253148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251592    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 22:04:31.253148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251598    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 22:04:31.253148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251602    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 22:04:31.253148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251606    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 22:04:31.253148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251610    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 22:04:31.253148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251615    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 22:04:31.253148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251619    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 22:04:31.253148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251623    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 22:04:31.253148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251632    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 22:04:31.253148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251636    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 22:04:31.253148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251642    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 22:04:31.253148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251646    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 22:04:31.253650 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251651    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 22:04:31.253650 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251655    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 22:04:31.253650 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251659    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 22:04:31.253650 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251664    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 22:04:31.253650 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251668    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 22:04:31.253650 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251673    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 22:04:31.253650 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251679    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 22:04:31.253650 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251683    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 22:04:31.253650 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251692    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 22:04:31.253650 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251696    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 22:04:31.253650 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251700    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 22:04:31.253650 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251704    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 22:04:31.253650 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251708    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 22:04:31.253650 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251712    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 22:04:31.253650 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251717    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 22:04:31.253650 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251721    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 22:04:31.253650 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251725    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 22:04:31.253650 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251729    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 22:04:31.253650 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.251733    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 22:04:31.253650 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252754    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 22:04:31.254163 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252765    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 22:04:31.254163 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252770    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 22:04:31.254163 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252774    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 22:04:31.254163 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252777    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 22:04:31.254163 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252780    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 22:04:31.254163 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252783    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 22:04:31.254163 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252786    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 22:04:31.254163 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252789    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 22:04:31.254163 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252791    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 22:04:31.254163 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252794    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 22:04:31.254163 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252797    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 22:04:31.254163 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252799    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 22:04:31.254163 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252802    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 22:04:31.254163 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252805    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 22:04:31.254163 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252807    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 22:04:31.254163 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252809    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 22:04:31.254163 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252813    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 22:04:31.254163 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252815    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 22:04:31.254163 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252817    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 22:04:31.254163 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252820    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 22:04:31.254689 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252822    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 22:04:31.254689 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252825    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 22:04:31.254689 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252828    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 22:04:31.254689 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252830    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 22:04:31.254689 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252834    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 22:04:31.254689 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252838    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 22:04:31.254689 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252841    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 22:04:31.254689 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252844    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 22:04:31.254689 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252847    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 22:04:31.254689 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252850    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 22:04:31.254689 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252852    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 22:04:31.254689 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252855    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 22:04:31.254689 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252858    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 22:04:31.254689 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252861    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 22:04:31.254689 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252864    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 22:04:31.254689 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252866    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 22:04:31.254689 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252869    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 22:04:31.254689 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252872    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 22:04:31.254689 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252875    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 22:04:31.255161 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252877    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 22:04:31.255161 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252880    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 22:04:31.255161 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252882    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 22:04:31.255161 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252885    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 22:04:31.255161 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252887    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 22:04:31.255161 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252892    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 22:04:31.255161 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252896    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 22:04:31.255161 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252898    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 22:04:31.255161 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252901    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 22:04:31.255161 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252904    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 22:04:31.255161 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252907    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 22:04:31.255161 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252910    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 22:04:31.255161 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252912    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 22:04:31.255161 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252915    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 22:04:31.255161 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252918    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 22:04:31.255161 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252920    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 22:04:31.255161 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252923    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 22:04:31.255161 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252926    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 22:04:31.255161 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252930    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 22:04:31.255644 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252933    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 22:04:31.255644 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252936    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 22:04:31.255644 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252939    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 22:04:31.255644 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252941    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 22:04:31.255644 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252944    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 22:04:31.255644 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252947    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 22:04:31.255644 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252950    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 22:04:31.255644 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252953    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 22:04:31.255644 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252955    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 22:04:31.255644 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252958    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 22:04:31.255644 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252961    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 22:04:31.255644 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252963    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 22:04:31.255644 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252966    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 22:04:31.255644 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252968    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 22:04:31.255644 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252971    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 22:04:31.255644 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252973    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 22:04:31.255644 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252977    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 22:04:31.255644 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252980    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 22:04:31.255644 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252983    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 22:04:31.255644 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252985    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 22:04:31.256122 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252988    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 22:04:31.256122 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252990    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 22:04:31.256122 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252993    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 22:04:31.256122 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252996    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 22:04:31.256122 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.252999    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 22:04:31.256122 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253002    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 22:04:31.256122 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253004    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 22:04:31.256122 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253007    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 22:04:31.256122 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253009    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 22:04:31.256122 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253011    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 22:04:31.256122 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253014    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 22:04:31.256122 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253016    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 22:04:31.256122 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253019    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 22:04:31.256122 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253021    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 22:04:31.256122 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253024    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 22:04:31.256122 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253029    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 22:04:31.256122 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253032    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 22:04:31.256122 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253034    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 22:04:31.256122 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253037    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 22:04:31.256122 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253040    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 22:04:31.256661 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253449    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 22:04:31.256661 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253455    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 22:04:31.256661 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253457    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 22:04:31.256661 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253460    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 22:04:31.256661 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253463    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 22:04:31.256661 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253466    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 22:04:31.256661 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253468    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 22:04:31.256661 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253471    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 22:04:31.256661 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253474    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 22:04:31.256661 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253477    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 22:04:31.256661 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253479    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 22:04:31.256661 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253482    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 22:04:31.256661 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253486    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 22:04:31.256661 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253490    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 22:04:31.256661 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253494    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 22:04:31.256661 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253497    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 22:04:31.256661 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253501    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 22:04:31.256661 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253503    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 22:04:31.256661 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253506    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 22:04:31.256661 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253509    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 22:04:31.257148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253511    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 22:04:31.257148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253514    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 22:04:31.257148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253516    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 22:04:31.257148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253519    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 22:04:31.257148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253522    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 22:04:31.257148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253524    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 22:04:31.257148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253527    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 22:04:31.257148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253534    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 22:04:31.257148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253536    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 22:04:31.257148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253539    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 22:04:31.257148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253542    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 22:04:31.257148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253545    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 22:04:31.257148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253547    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 22:04:31.257148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253550    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 22:04:31.257148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253553    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 22:04:31.257148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253556    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 22:04:31.257148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253558    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 22:04:31.257148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253561    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 22:04:31.257148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253564    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 22:04:31.257148 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253566    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 22:04:31.257652 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253569    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 22:04:31.257652 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253572    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 22:04:31.257652 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253574    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 22:04:31.257652 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253577    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 22:04:31.257652 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253579    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 22:04:31.257652 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253582    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 22:04:31.257652 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253584    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 22:04:31.257652 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253587    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 22:04:31.257652 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253589    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 22:04:31.257652 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253598    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 22:04:31.257652 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253601    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 22:04:31.257652 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253604    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 22:04:31.257652 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253606    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 22:04:31.257652 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253609    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 22:04:31.257652 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253612    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 22:04:31.257652 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253614    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 22:04:31.257652 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253617    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 22:04:31.257652 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253619    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 22:04:31.257652 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253622    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 22:04:31.257652 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253625    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 22:04:31.258167 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253628    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 22:04:31.258167 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253631    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 22:04:31.258167 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253634    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 22:04:31.258167 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253636    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 22:04:31.258167 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253639    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 22:04:31.258167 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253641    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 22:04:31.258167 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253644    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 22:04:31.258167 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253647    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 22:04:31.258167 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253649    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 22:04:31.258167 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253652    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 22:04:31.258167 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253654    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 22:04:31.258167 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253657    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 22:04:31.258167 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253660    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 22:04:31.258167 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253663    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 22:04:31.258167 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253665    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 22:04:31.258167 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253668    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 22:04:31.258167 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253670    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 22:04:31.258167 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253673    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 22:04:31.258167 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253675    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 22:04:31.258167 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253678    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 22:04:31.258681 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253680    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 22:04:31.258681 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253683    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 22:04:31.258681 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253685    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 22:04:31.258681 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253688    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 22:04:31.258681 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253690    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 22:04:31.258681 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253694    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 22:04:31.258681 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253697    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 22:04:31.258681 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253699    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 22:04:31.258681 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253703    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 22:04:31.258681 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253706    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 22:04:31.258681 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253709    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 22:04:31.258681 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253714    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 22:04:31.258681 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253717    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 22:04:31.258681 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253720    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 22:04:31.258681 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253722    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 22:04:31.258681 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253725    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 22:04:31.258681 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253728    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 22:04:31.258681 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253730    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 22:04:31.258681 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.253733    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 22:04:31.258681 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253807    2574 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253814    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253821    2574 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253826    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253833    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253836    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253840    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253845    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253848    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253851    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253854    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253858    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253861    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253864    2574 flags.go:64] FLAG: --cgroup-root=""
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253867    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253870    2574 flags.go:64] FLAG: --client-ca-file=""
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253873    2574 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253877    2574 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253881    2574 flags.go:64] FLAG: --cluster-domain=""
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253884    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253887    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253890    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253893    2574 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253901    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 22:04:31.259292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253905    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253908    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253912    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253915    2574 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253919    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253922    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253925    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253928    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253932    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253936    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253939    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253941    2574 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253944    2574 flags.go:64] FLAG: --enable-server="true"
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253947    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253953    2574 flags.go:64] FLAG: --event-burst="100"
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253956    2574 flags.go:64] FLAG: --event-qps="50"
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253959    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253962    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253965    2574 flags.go:64] FLAG: --eviction-hard=""
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253968    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253971    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253974    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253977    2574 flags.go:64] FLAG: --eviction-soft=""
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253980    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253983    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 22:04:31.259864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253986    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253989    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253992    2574 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253995    2574 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.253997    2574 flags.go:64] FLAG: --feature-gates=""
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254001    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254004    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254009    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254012    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254015    2574 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254018    2574 flags.go:64] FLAG: --help="false"
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254021    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-143-69.ec2.internal"
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254024    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254027    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254030    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254033    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254037    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254040    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254043    2574 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254045    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254048    2574 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254051    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254054    2574 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254057    2574 flags.go:64] FLAG: --kube-reserved=""
Jul 28 22:04:31.260521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254060    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254062    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254065    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254068    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254071    2574 flags.go:64] FLAG: --lock-file=""
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254073    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254076    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254079    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254084    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254088    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254090    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254093    2574 flags.go:64] FLAG: --logging-format="text"
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254096    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254099    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254102    2574 flags.go:64] FLAG: --manifest-url=""
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254106    2574 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254110    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254113    2574 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254120    2574 flags.go:64] FLAG: --max-pods="110"
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254123    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254126    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254129    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254132    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254135    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254138    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 22:04:31.261103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254141    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254148    2574 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254151    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254154    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254157    2574 flags.go:64] FLAG: --pod-cidr=""
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254160    2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254166    2574 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254169    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254172    2574 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254175    2574 flags.go:64] FLAG: --port="10250"
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254178    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254181    2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04b4871397ac76427"
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254185    2574 flags.go:64] FLAG: --qos-reserved=""
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254188    2574 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254190    2574 flags.go:64] FLAG: --register-node="true"
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254193    2574 flags.go:64] FLAG: --register-with-taints=""
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254197    2574 flags.go:64] FLAG: --registry-burst="10"
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254201    2574 flags.go:64] FLAG: --registry-qps="5"
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254204    2574 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254207    2574 flags.go:64] FLAG: --reserved-memory=""
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254211    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254213    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254217    2574 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254220    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254223    2574 flags.go:64] FLAG: --runonce="false"
Jul 28 22:04:31.261802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254226    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254245    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254248    2574 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254252    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254255    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254258    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254261    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254264    2574 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254267    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254270    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254273    2574 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254275    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254278    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254281    2574 flags.go:64] FLAG: --system-cgroups=""
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254284    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254290    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254292    2574 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254295    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254299    2574 flags.go:64] FLAG: --tls-min-version=""
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254302    2574 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254305    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254308    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254311    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254314    2574 flags.go:64] FLAG: --v="2"
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254318    2574 flags.go:64] FLAG: --version="false"
Jul 28 22:04:31.262437 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254322    2574 flags.go:64] FLAG: --vmodule=""
Jul 28 22:04:31.263050 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254326    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 22:04:31.263050 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.254329    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 22:04:31.263050 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254454    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 22:04:31.263050 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254458    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 22:04:31.263050 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254462    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 22:04:31.263050 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254464    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 22:04:31.263050 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254467    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 22:04:31.263050 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254471    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 22:04:31.263050 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254474    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 22:04:31.263050 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254476    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 22:04:31.263050 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254479    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 22:04:31.263050 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254482    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 22:04:31.263050 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254485    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 22:04:31.263050 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254487    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 22:04:31.263050 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254490    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 22:04:31.263050 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254493    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 22:04:31.263050 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254495    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 22:04:31.263050 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254499    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 22:04:31.263050 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254503    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 22:04:31.263050 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254506    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 22:04:31.263610 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254509    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 22:04:31.263610 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254512    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 22:04:31.263610 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254515    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 22:04:31.263610 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254517    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 22:04:31.263610 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254520    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 22:04:31.263610 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254523    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 22:04:31.263610 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254525    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 22:04:31.263610 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254528    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 22:04:31.263610 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254530    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 22:04:31.263610 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254533    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 22:04:31.263610 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254536    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 22:04:31.263610 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254538    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 22:04:31.263610 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254542    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 22:04:31.263610 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254544    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 22:04:31.263610 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254547    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 22:04:31.263610 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254551    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 22:04:31.263610 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254554    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 22:04:31.263610 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254556    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 22:04:31.263610 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254559    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 22:04:31.264066 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254563    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 22:04:31.264066 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254565    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 22:04:31.264066 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254568    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 22:04:31.264066 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254571    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 22:04:31.264066 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254573    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 22:04:31.264066 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254576    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 22:04:31.264066 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254578    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 22:04:31.264066 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254581    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 22:04:31.264066 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254583    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 22:04:31.264066 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254586    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 22:04:31.264066 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254589    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 22:04:31.264066 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254591    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 22:04:31.264066 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254594    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 22:04:31.264066 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254596    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 22:04:31.264066 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254599    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 22:04:31.264066 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254602    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 22:04:31.264066 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254604    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 22:04:31.264066 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254607    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 22:04:31.264066 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254618    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 22:04:31.264066 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254621    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 22:04:31.264566 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254624    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 22:04:31.264566 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254626    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 22:04:31.264566 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254629    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 22:04:31.264566 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254632    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 22:04:31.264566 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254634    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 22:04:31.264566 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254637    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 22:04:31.264566 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254640    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 22:04:31.264566 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254644    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 22:04:31.264566 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254649    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 22:04:31.264566 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254653    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 22:04:31.264566 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254656    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 22:04:31.264566 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254659    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 22:04:31.264566 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254663    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 22:04:31.264566 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254666    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 22:04:31.264566 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254669    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 22:04:31.264566 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254671    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 22:04:31.264566 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254674    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 22:04:31.264566 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254677    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 22:04:31.264566 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254679    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 22:04:31.264566 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254682    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 22:04:31.265067 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254684    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 22:04:31.265067 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254689    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 22:04:31.265067 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254692    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 22:04:31.265067 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254695    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 22:04:31.265067 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254698    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 22:04:31.265067 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254700    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 22:04:31.265067 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254703    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 22:04:31.265067 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254706    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 22:04:31.265067 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254709    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 22:04:31.265067 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254711    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 22:04:31.265067 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254714    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 22:04:31.265067 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254717    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 22:04:31.265067 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254719    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 22:04:31.265067 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254722    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 22:04:31.265067 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254724    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 22:04:31.265067 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254727    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 22:04:31.265067 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254730    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 22:04:31.265067 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254732    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 22:04:31.265067 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254735    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 22:04:31.265539 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254739    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 22:04:31.265539 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254742    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 22:04:31.265539 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.254745    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 22:04:31.265539 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.255311    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 22:04:31.265539 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.262023    2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 22:04:31.265539 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.262039    2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 22:04:31.265539 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.262698    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 22:04:31.265539 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.262708    2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 22:04:31.265539 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262770    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 22:04:31.265539 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262775    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 22:04:31.265539 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262779    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 22:04:31.265539 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262782    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 22:04:31.265539 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262785    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 22:04:31.265539 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262788    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 22:04:31.265539 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262791    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 22:04:31.265923 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262794    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 22:04:31.265923 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262797    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 22:04:31.265923 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262801    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 22:04:31.265923 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262805    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 22:04:31.265923 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262808    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 22:04:31.265923 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262811    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 22:04:31.265923 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262815    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 22:04:31.265923 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262818    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 22:04:31.265923 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262821    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 22:04:31.265923 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262823    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 22:04:31.265923 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262826    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 22:04:31.265923 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262828    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 22:04:31.265923 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262831    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 22:04:31.265923 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262834    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 22:04:31.265923 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262836    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 22:04:31.265923 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262839    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 22:04:31.265923 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262841    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 22:04:31.265923 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262844    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 22:04:31.265923 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262847    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 22:04:31.265923 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262850    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 22:04:31.266420 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262852    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 22:04:31.266420 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262855    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 22:04:31.266420 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262859    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 22:04:31.266420 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262862    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 22:04:31.266420 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262865    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 22:04:31.266420 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262867    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 22:04:31.266420 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262870    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 22:04:31.266420 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262873    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 22:04:31.266420 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262875    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 22:04:31.266420 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262878    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 22:04:31.266420 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262881    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 22:04:31.266420 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262883    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 22:04:31.266420 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262886    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 22:04:31.266420 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262888    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 22:04:31.266420 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262891    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 22:04:31.266420 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262894    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 22:04:31.266420 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262896    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 22:04:31.266420 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262899    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 22:04:31.266420 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262901    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 22:04:31.266420 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262904    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 22:04:31.266930 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262906    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 22:04:31.266930 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262909    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 22:04:31.266930 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262912    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 22:04:31.266930 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262914    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 22:04:31.266930 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262917    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 22:04:31.266930 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262920    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 22:04:31.266930 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262922    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 22:04:31.266930 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262924    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 22:04:31.266930 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262927    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 22:04:31.266930 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262930    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 22:04:31.266930 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262932    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 22:04:31.266930 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262935    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 22:04:31.266930 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262938    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 22:04:31.266930 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262940    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 22:04:31.266930 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262944    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 22:04:31.266930 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262955    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 22:04:31.266930 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262957    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 22:04:31.266930 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262960    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 22:04:31.266930 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262963    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 22:04:31.266930 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262965    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 22:04:31.267433 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262968    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 22:04:31.267433 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262970    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 22:04:31.267433 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262973    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 22:04:31.267433 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262975    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 22:04:31.267433 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262978    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 22:04:31.267433 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262980    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 22:04:31.267433 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262983    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 22:04:31.267433 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262985    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 22:04:31.267433 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262987    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 22:04:31.267433 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262991    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 22:04:31.267433 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262995    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 22:04:31.267433 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.262998    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 22:04:31.267433 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263000    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 22:04:31.267433 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263003    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 22:04:31.267433 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263005    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 22:04:31.267433 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263008    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 22:04:31.267433 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263011    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 22:04:31.267433 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263013    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 22:04:31.267433 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263016    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 22:04:31.267891 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263019    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 22:04:31.267891 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263021    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 22:04:31.267891 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263024    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 22:04:31.267891 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263027    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 22:04:31.267891 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263029    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 22:04:31.267891 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263032    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 22:04:31.267891 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263036    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 22:04:31.267891 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263040    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 22:04:31.267891 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263043    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 22:04:31.267891 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263051    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 22:04:31.267891 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263054    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 22:04:31.267891 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263057    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 22:04:31.267891 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263060    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 22:04:31.267891 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.263065    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 22:04:31.267891 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263184    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 22:04:31.268303 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263189    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 22:04:31.268303 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263192    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 22:04:31.268303 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263195    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 22:04:31.268303 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263198    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 22:04:31.268303 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263201    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 22:04:31.268303 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263204    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 22:04:31.268303 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263207    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 22:04:31.268303 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263210    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 22:04:31.268303 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263214    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 22:04:31.268303 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263218    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 22:04:31.268303 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263220    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 22:04:31.268303 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263223    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 22:04:31.268303 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263226    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 22:04:31.268303 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263243    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 22:04:31.268303 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263246    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 22:04:31.268303 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263249    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 22:04:31.268303 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263252    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 22:04:31.268303 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263255    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 22:04:31.268303 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263258    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 22:04:31.268763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263260    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 22:04:31.268763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263263    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 22:04:31.268763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263266    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 22:04:31.268763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263269    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 22:04:31.268763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263271    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 22:04:31.268763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263274    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 22:04:31.268763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263276    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 22:04:31.268763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263278    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 22:04:31.268763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263282    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 22:04:31.268763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263286    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 22:04:31.268763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263289    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 22:04:31.268763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263292    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 22:04:31.268763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263295    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 22:04:31.268763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263298    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 22:04:31.268763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263301    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 22:04:31.268763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263303    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 22:04:31.268763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263306    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 22:04:31.268763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263308    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 22:04:31.268763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263311    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 22:04:31.269274 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263314    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 22:04:31.269274 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263316    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 22:04:31.269274 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263319    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 22:04:31.269274 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263322    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 22:04:31.269274 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263324    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 22:04:31.269274 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263326    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 22:04:31.269274 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263329    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 22:04:31.269274 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263331    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 22:04:31.269274 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263334    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 22:04:31.269274 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263336    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 22:04:31.269274 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263339    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 22:04:31.269274 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263342    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 22:04:31.269274 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263345    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 22:04:31.269274 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263347    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 22:04:31.269274 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263350    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 22:04:31.269274 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263352    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 22:04:31.269274 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263355    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 22:04:31.269274 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263358    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 22:04:31.269274 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263360    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 22:04:31.269274 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263363    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 22:04:31.269763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263365    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 22:04:31.269763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263367    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 22:04:31.269763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263370    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 22:04:31.269763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263373    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 22:04:31.269763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263375    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 22:04:31.269763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263378    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 22:04:31.269763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263381    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 22:04:31.269763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263383    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 22:04:31.269763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263386    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 22:04:31.269763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263388    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 22:04:31.269763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263391    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 22:04:31.269763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263393    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 22:04:31.269763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263396    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 22:04:31.269763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263398    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 22:04:31.269763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263401    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 22:04:31.269763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263403    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 22:04:31.269763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263405    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 22:04:31.269763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263409    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 22:04:31.269763 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263412    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 22:04:31.270276 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263415    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 22:04:31.270276 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263418    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 22:04:31.270276 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263420    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 22:04:31.270276 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263422    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 22:04:31.270276 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263426    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 22:04:31.270276 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263428    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 22:04:31.270276 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263431    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 22:04:31.270276 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263433    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 22:04:31.270276 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263436    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 22:04:31.270276 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263438    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 22:04:31.270276 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263441    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 22:04:31.270276 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263443    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 22:04:31.270276 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263446    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 22:04:31.270276 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263448    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 22:04:31.270276 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263451    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 22:04:31.270276 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263453    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 22:04:31.270276 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263456    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 22:04:31.270276 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263458    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 22:04:31.270276 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263461    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 22:04:31.270276 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263463    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 22:04:31.270815 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:31.263466    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 22:04:31.270815 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.263471    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 22:04:31.270815 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.263600    2574 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 22:04:31.270815 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.265480    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 22:04:31.270815 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.266594    2574 server.go:1019] "Starting client certificate rotation"
Jul 28 22:04:31.270815 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.266708    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 22:04:31.270815 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.267401    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 22:04:31.270815 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.268904    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 22:04:31.270815 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.270476    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 22:04:31.281144 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.281120    2574 log.go:25] "Validated CRI v1 runtime API"
Jul 28 22:04:31.284689 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.284668    2574 log.go:25] "Validated CRI v1 image API"
Jul 28 22:04:31.285456 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.285432    2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 22:04:31.287615 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.287596    2574 fs.go:135] Filesystem UUIDs: map[4b12fda3-44a7-432f-9841-75cf707f30e4:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 fe645411-939b-4591-8f2d-46439742da2b:/dev/nvme0n1p4]
Jul 28 22:04:31.287679 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.287615    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 22:04:31.293486 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.293378    2574 manager.go:216] Machine: {Timestamp:2026-07-28 22:04:31.291558054 +0000 UTC m=+0.275700665 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3119032 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b9b71c2536b2d2566d7cabf98a887 SystemUUID:ec2b9b71-c253-6b2d-2566-d7cabf98a887 BootID:32ea2aba-114e-4c2b-9486-e6c270e5b3a3 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b6:2d:43:83:77 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b6:2d:43:83:77 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:91:fc:8e:cc:81 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 22:04:31.293486 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.293480    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 22:04:31.293596 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.293563    2574 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 22:04:31.295416 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.295398    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 22:04:31.296257 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.296224    2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 22:04:31.296396 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.296260    2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-69.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 22:04:31.296840 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.296831    2574 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 22:04:31.296871 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.296842    2574 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 22:04:31.296871 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.296849    2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 22:04:31.297362 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.297351    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 22:04:31.298207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.298198    2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 22:04:31.300026 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.300015    2574 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 22:04:31.300152 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.300143    2574 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 22:04:31.300181 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.300174    2574 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 22:04:31.300211 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.300191    2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 22:04:31.300211 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.300209    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 22:04:31.300288 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.300221    2574 kubelet.go:398] "Adding apiserver pod source"
Jul 28 22:04:31.300288 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.300284    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 22:04:31.300807 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.300794    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 22:04:31.300864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.300815    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 22:04:31.302666 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.302651    2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 22:04:31.304276 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.304264    2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 22:04:31.304331 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.304293    2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 22:04:31.306132 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.306114    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 22:04:31.306212 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.306140    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 22:04:31.306212 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.306149    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 22:04:31.306212 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.306158    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 22:04:31.306212 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.306176    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 22:04:31.306212 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.306187    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 22:04:31.306212 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.306200    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 22:04:31.306212 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.306210    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 22:04:31.306452 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.306224    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 22:04:31.306452 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.306246    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 22:04:31.306452 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.306275    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 22:04:31.306452 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.306288    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 22:04:31.307690 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.307676    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 22:04:31.307690 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.307691    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 22:04:31.310983 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.310967    2574 server.go:1268] "Started kubelet"
Jul 28 22:04:31.311157 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.311077    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 22:04:31.311305 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.311283    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 22:04:31.311398 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.311349    2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 22:04:31.311888 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:31.311840    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 22:04:31.311963 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:31.311912    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-69.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 22:04:31.311992 ip-10-0-143-69 systemd[1]: Started Kubernetes Kubelet.
Jul 28 22:04:31.314107 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.312061    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-69.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 22:04:31.314586 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.314564    2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 22:04:31.316388 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:31.316366    2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 22:04:31.317517 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.317501    2574 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 22:04:31.318209 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.318190    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ddxrn"
Jul 28 22:04:31.318834 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:31.318051    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-69.ec2.internal.18c6934f596c191a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-69.ec2.internal,UID:ip-10-0-143-69.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-69.ec2.internal,},FirstTimestamp:2026-07-28 22:04:31.310936346 +0000 UTC m=+0.295078957,LastTimestamp:2026-07-28 22:04:31.310936346 +0000 UTC m=+0.295078957,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-69.ec2.internal,}"
Jul 28 22:04:31.320872 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.320852    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 22:04:31.320938 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.320857    2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 22:04:31.321002 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:31.320980    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-69.ec2.internal\" not found"
Jul 28 22:04:31.321429 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.321412    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 22:04:31.321429 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.321416    2574 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 22:04:31.321566 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.321439    2574 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 22:04:31.321619 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.321592    2574 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 22:04:31.321619 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.321603    2574 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 22:04:31.321706 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.321660    2574 factory.go:55] Registering systemd factory
Jul 28 22:04:31.321706 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.321678    2574 factory.go:223] Registration of the systemd container factory successfully
Jul 28 22:04:31.321924 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.321909    2574 factory.go:153] Registering CRI-O factory
Jul 28 22:04:31.321924 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.321925    2574 factory.go:223] Registration of the crio container factory successfully
Jul 28 22:04:31.322053 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.321984    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 22:04:31.322053 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.322004    2574 factory.go:103] Registering Raw factory
Jul 28 22:04:31.322053 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.322017    2574 manager.go:1196] Started watching for new ooms in manager
Jul 28 22:04:31.322155 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:31.322080    2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-69.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 22:04:31.322261 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:31.322223    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 22:04:31.322500 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.322489    2574 manager.go:349] Starting recovery of all containers
Jul 28 22:04:31.325361 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.325334    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ddxrn"
Jul 28 22:04:31.333626 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.333609    2574 manager.go:354] Recovery completed
Jul 28 22:04:31.337939 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.337923    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 22:04:31.341680 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.341664    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-69.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 22:04:31.341748 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.341694    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-69.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 22:04:31.341748 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.341707    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-69.ec2.internal" event="NodeHasSufficientPID"
Jul 28 22:04:31.342285 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.342267    2574 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 22:04:31.342285 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.342286    2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 22:04:31.342409 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.342296    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 22:04:31.342409 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.342303    2574 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 22:04:31.342999 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.342984    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-69.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 22:04:31.343061 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.343011    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-69.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 22:04:31.343061 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.343022    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-69.ec2.internal" event="NodeHasSufficientPID"
Jul 28 22:04:31.343766 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.343754    2574 policy_none.go:49] "None policy: Start"
Jul 28 22:04:31.343808 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.343771    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 22:04:31.343808 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.343783    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 22:04:31.344778 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.344769    2574 policy_none.go:47] "Start"
Jul 28 22:04:31.385063 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.384891    2574 manager.go:337] "Starting Device Plugin manager"
Jul 28 22:04:31.385193 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:31.385090    2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 22:04:31.385193 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.385105    2574 server.go:87] "Starting device plugin registration server"
Jul 28 22:04:31.385427 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.385412    2574 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 22:04:31.385469 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.385430    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 22:04:31.385534 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.385515    2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 22:04:31.385633 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.385615    2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 22:04:31.385633 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.385628    2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 22:04:31.386172 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:31.386149    2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 22:04:31.386275 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:31.386202    2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-69.ec2.internal\" not found"
Jul 28 22:04:31.485650 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.485627    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 22:04:31.485870 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.485625    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 22:04:31.486717 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.486695    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-69.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 22:04:31.486804 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.486739    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-69.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 22:04:31.486804 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.486755    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-69.ec2.internal" event="NodeHasSufficientPID"
Jul 28 22:04:31.486804 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.486795    2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-69.ec2.internal"
Jul 28 22:04:31.487130 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.487115    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 22:04:31.487203 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.487156    2574 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 22:04:31.487203 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.487184    2574 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 22:04:31.487321 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:31.487228    2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 22:04:31.489346 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.489328    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 22:04:31.492667 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.492646    2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-69.ec2.internal"
Jul 28 22:04:31.492750 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:31.492669    2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-69.ec2.internal\": node \"ip-10-0-143-69.ec2.internal\" not found"
Jul 28 22:04:31.518793 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:31.518711    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-69.ec2.internal\" not found"
Jul 28 22:04:31.587696 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.587634    2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-69.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-69.ec2.internal"]
Jul 28 22:04:31.587802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.587744    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 22:04:31.588874 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.588857    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-69.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 22:04:31.588953 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.588889    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-69.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 22:04:31.588953 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.588904    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-69.ec2.internal" event="NodeHasSufficientPID"
Jul 28 22:04:31.590133 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.590121    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 22:04:31.590256 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.590242    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-69.ec2.internal"
Jul 28 22:04:31.590312 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.590271    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 22:04:31.590983 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.590968    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-69.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 22:04:31.591071 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.590996    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-69.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 22:04:31.591071 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.591010    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-69.ec2.internal" event="NodeHasSufficientPID"
Jul 28 22:04:31.591517 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.591502    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-69.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 22:04:31.591588 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.591527    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-69.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 22:04:31.591588 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.591537    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-69.ec2.internal" event="NodeHasSufficientPID"
Jul 28 22:04:31.592508 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.592495    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-69.ec2.internal"
Jul 28 22:04:31.592568 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.592518    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 22:04:31.593170 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.593154    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-69.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 22:04:31.593262 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.593181    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-69.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 22:04:31.593262 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.593203    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-69.ec2.internal" event="NodeHasSufficientPID"
Jul 28 22:04:31.609821 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:31.609795    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-69.ec2.internal\" not found" node="ip-10-0-143-69.ec2.internal"
Jul 28 22:04:31.613331 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:31.613315    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-69.ec2.internal\" not found" node="ip-10-0-143-69.ec2.internal"
Jul 28 22:04:31.619366 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:31.619345    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-69.ec2.internal\" not found"
Jul 28 22:04:31.622929 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.622910    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/54f8ab6a0dce2d047f0664fad06551cb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-69.ec2.internal\" (UID: \"54f8ab6a0dce2d047f0664fad06551cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-69.ec2.internal"
Jul 28 22:04:31.623011 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.622940    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/54f8ab6a0dce2d047f0664fad06551cb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-69.ec2.internal\" (UID: \"54f8ab6a0dce2d047f0664fad06551cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-69.ec2.internal"
Jul 28 22:04:31.623011 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.622968    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7cc3a9f08dccc44aae468666893de9e2-config\") pod \"kube-apiserver-proxy-ip-10-0-143-69.ec2.internal\" (UID: \"7cc3a9f08dccc44aae468666893de9e2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-69.ec2.internal"
Jul 28 22:04:31.719462 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:31.719430    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-69.ec2.internal\" not found"
Jul 28 22:04:31.723827 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.723808    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/54f8ab6a0dce2d047f0664fad06551cb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-69.ec2.internal\" (UID: \"54f8ab6a0dce2d047f0664fad06551cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-69.ec2.internal"
Jul 28 22:04:31.723893 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.723837    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/54f8ab6a0dce2d047f0664fad06551cb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-69.ec2.internal\" (UID: \"54f8ab6a0dce2d047f0664fad06551cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-69.ec2.internal"
Jul 28 22:04:31.723893 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.723856    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7cc3a9f08dccc44aae468666893de9e2-config\") pod \"kube-apiserver-proxy-ip-10-0-143-69.ec2.internal\" (UID: \"7cc3a9f08dccc44aae468666893de9e2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-69.ec2.internal"
Jul 28 22:04:31.723957 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.723899    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/54f8ab6a0dce2d047f0664fad06551cb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-69.ec2.internal\" (UID: \"54f8ab6a0dce2d047f0664fad06551cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-69.ec2.internal"
Jul 28 22:04:31.723957 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.723902    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/54f8ab6a0dce2d047f0664fad06551cb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-69.ec2.internal\" (UID: \"54f8ab6a0dce2d047f0664fad06551cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-69.ec2.internal"
Jul 28 22:04:31.724019 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.723902    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7cc3a9f08dccc44aae468666893de9e2-config\") pod \"kube-apiserver-proxy-ip-10-0-143-69.ec2.internal\" (UID: \"7cc3a9f08dccc44aae468666893de9e2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-69.ec2.internal"
Jul 28 22:04:31.819858 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:31.819772    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-69.ec2.internal\" not found"
Jul 28 22:04:31.911841 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.911801    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-69.ec2.internal"
Jul 28 22:04:31.916684 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:31.916667    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-69.ec2.internal"
Jul 28 22:04:31.920292 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:31.920275    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-69.ec2.internal\" not found"
Jul 28 22:04:32.020829 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:32.020795    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-69.ec2.internal\" not found"
Jul 28 22:04:32.121626 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:32.121525    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-69.ec2.internal\" not found"
Jul 28 22:04:32.222041 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:32.222009    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-69.ec2.internal\" not found"
Jul 28 22:04:32.225631 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.225614    2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 22:04:32.266757 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.266732    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 22:04:32.267263 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.266884    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 22:04:32.267263 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.266900    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 22:04:32.301107 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.301082    2574 apiserver.go:52] "Watching apiserver"
Jul 28 22:04:32.317001 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.316973    2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 22:04:32.318532 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.318509    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-bzhk8","openshift-cluster-node-tuning-operator/tuned-97vh9","openshift-image-registry/node-ca-gnhk8","openshift-multus/multus-5vhbh","openshift-network-diagnostics/network-check-target-np5j9","openshift-network-operator/iptables-alerter-26gxp","openshift-ovn-kubernetes/ovnkube-node-jrh49","kube-system/global-pull-secret-syncer-qw7gj","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d","openshift-dns/node-resolver-swmbg","openshift-multus/multus-additional-cni-plugins-mj477","openshift-multus/network-metrics-daemon-dvqcn"]
Jul 28 22:04:32.319748 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.319730    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bzhk8"
Jul 28 22:04:32.320661 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.320643    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.320965 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.320949    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 22:04:32.321088 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.321075    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-69.ec2.internal"
Jul 28 22:04:32.321603 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.321587    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 22:04:32.321687 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.321597    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8c8hf\""
Jul 28 22:04:32.321687 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.321680    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 22:04:32.321774 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.321706    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gnhk8"
Jul 28 22:04:32.323611 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.323253    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 22:04:32.323611 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.323323    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 22:04:32.323611 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.323388    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.323810 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.323684    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 22:04:32.324749 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.324730    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:04:32.324867 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:32.324844    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-np5j9" podUID="8497aa3c-a69a-4e64-a1f3-306cd512ae81"
Jul 28 22:04:32.324929 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.324899    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 22:04:32.325006 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.324987    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wdbc4\""
Jul 28 22:04:32.325190 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.325171    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 22:04:32.325442 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.325425    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-95vz5\""
Jul 28 22:04:32.325731 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.325714    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 22:04:32.325944 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.325899    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-host\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.326036 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.325961    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0efef71b-3b20-4214-914d-5138f89c14fc-host\") pod \"node-ca-gnhk8\" (UID: \"0efef71b-3b20-4214-914d-5138f89c14fc\") " pod="openshift-image-registry/node-ca-gnhk8"
Jul 28 22:04:32.326036 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.326001    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c6e1d829-065c-4516-a1cf-456e7d3ff8c1-agent-certs\") pod \"konnectivity-agent-bzhk8\" (UID: \"c6e1d829-065c-4516-a1cf-456e7d3ff8c1\") " pod="kube-system/konnectivity-agent-bzhk8"
Jul 28 22:04:32.326143 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.326033    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-etc-modprobe-d\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.326143 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.326094    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-p7r8b\""
Jul 28 22:04:32.326311 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.326129    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-etc-sysctl-conf\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.326311 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.326257    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-var-lib-kubelet\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.326445 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.326307    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0efef71b-3b20-4214-914d-5138f89c14fc-serviceca\") pod \"node-ca-gnhk8\" (UID: \"0efef71b-3b20-4214-914d-5138f89c14fc\") " pod="openshift-image-registry/node-ca-gnhk8"
Jul 28 22:04:32.326445 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.326319    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 22:04:32.326445 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.326351    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcl4k\" (UniqueName: \"kubernetes.io/projected/0efef71b-3b20-4214-914d-5138f89c14fc-kube-api-access-lcl4k\") pod \"node-ca-gnhk8\" (UID: \"0efef71b-3b20-4214-914d-5138f89c14fc\") " pod="openshift-image-registry/node-ca-gnhk8"
Jul 28 22:04:32.326445 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.326382    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-etc-sysconfig\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.326445 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.326391    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 22:04:32.326685 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.326444    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-sys\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.326685 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.326486    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-lib-modules\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.326685 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.326544    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b505b3f1-0db6-4d26-94c8-fdc9637e3415-tmp\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.326685 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.326624    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c6e1d829-065c-4516-a1cf-456e7d3ff8c1-konnectivity-ca\") pod \"konnectivity-agent-bzhk8\" (UID: \"c6e1d829-065c-4516-a1cf-456e7d3ff8c1\") " pod="kube-system/konnectivity-agent-bzhk8"
Jul 28 22:04:32.326877 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.326703    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-etc-sysctl-d\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.326877 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.326746    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-run\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.326877 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.326781    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b505b3f1-0db6-4d26-94c8-fdc9637e3415-etc-tuned\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.326877 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.326819    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr848\" (UniqueName: \"kubernetes.io/projected/b505b3f1-0db6-4d26-94c8-fdc9637e3415-kube-api-access-tr848\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.327067 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.326878    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-etc-kubernetes\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.327067 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.326911    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-etc-systemd\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.327067 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.326960    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 22:04:32.327204 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.327139    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-26gxp"
Jul 28 22:04:32.328897 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.328874    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 22:04:32.329128 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.329078    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 21:59:31 +0000 UTC" deadline="2028-03-30 22:43:27.98371761 +0000 UTC"
Jul 28 22:04:32.329209 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.329126    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.329293 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.329210    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7wbrc\""
Jul 28 22:04:32.329293 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.329228    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 22:04:32.329293 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.329260    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 22:04:32.329467 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.329129    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14664h38m55.654590722s"
Jul 28 22:04:32.329567 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.329549    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 22:04:32.329638 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.329631    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-69.ec2.internal"
Jul 28 22:04:32.330593 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.330577    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:32.330673 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:32.330642    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw7gj" podUID="6bbf80a3-a5e2-49a5-ad56-a21747cc4a16"
Jul 28 22:04:32.331012 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.330998    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lsxdk\""
Jul 28 22:04:32.331088 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.330999    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 22:04:32.331963 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.331941    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 22:04:32.332046 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.331977    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 22:04:32.332046 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.331983    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 22:04:32.332046 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.331997    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 22:04:32.332182 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.332089    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 22:04:32.332386 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.332372    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.332832 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.332816    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 22:04:32.334075 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.334057    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-swmbg"
Jul 28 22:04:32.334147 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.334119    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 22:04:32.335177 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.334808    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 22:04:32.335177 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.334862    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-p794c\""
Jul 28 22:04:32.335177 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.334925    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 22:04:32.335894 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.335879    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.336054 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.336036    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-499ng\""
Jul 28 22:04:32.336224 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.336207    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 22:04:32.336321 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.336281    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 22:04:32.337566 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.337545    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-143-69.ec2.internal"]
Jul 28 22:04:32.337627 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.337568    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zclqn\""
Jul 28 22:04:32.337627 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.337621    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:32.337722 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:32.337664    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvqcn" podUID="f738f750-e0f4-49e7-b7e6-0c151e664ce9"
Jul 28 22:04:32.337776 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.337759    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 22:04:32.337831 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.337773    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 22:04:32.340044 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.340028    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 22:04:32.340507 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.340477    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-69.ec2.internal"]
Jul 28 22:04:32.358274 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.358251    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rj7zc"
Jul 28 22:04:32.364252 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.364224    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rj7zc"
Jul 28 22:04:32.422940 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.422917    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 22:04:32.427752 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.427730    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-lib-modules\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.427864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.427763    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-etc-openvswitch\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.427864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.427793    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.427864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.427820    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-etc-sysctl-d\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.427981 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.427865    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-run\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.427981 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.427886    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-lib-modules\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.427981 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.427925    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b505b3f1-0db6-4d26-94c8-fdc9637e3415-etc-tuned\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.427981 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.427937    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-run\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.428159 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.427983    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr848\" (UniqueName: \"kubernetes.io/projected/b505b3f1-0db6-4d26-94c8-fdc9637e3415-kube-api-access-tr848\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.428159 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.427989    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-etc-sysctl-d\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.428159 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428032    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.428159 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428133    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-host-var-lib-kubelet\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.428497 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428158    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg85w\" (UniqueName: \"kubernetes.io/projected/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-kube-api-access-kg85w\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.428497 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428200    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-etc-kubernetes\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.428497 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428200    2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 22:04:32.428497 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428270    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-etc-kubernetes\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.428497 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428294    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7a52716-3aa2-425f-bd5b-40184621f385-host-slash\") pod \"iptables-alerter-26gxp\" (UID: \"d7a52716-3aa2-425f-bd5b-40184621f385\") " pod="openshift-network-operator/iptables-alerter-26gxp"
Jul 28 22:04:32.428497 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428334    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-host-slash\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.428497 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428363    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62w85\" (UniqueName: \"kubernetes.io/projected/435f03fc-de40-43d5-a9c9-7f22fb70825a-kube-api-access-62w85\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.428497 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428392    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fa48de0c-c971-487e-8bb8-c5e500ea2a33-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s9m6d\" (UID: \"fa48de0c-c971-487e-8bb8-c5e500ea2a33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.428497 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428419    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fa48de0c-c971-487e-8bb8-c5e500ea2a33-registration-dir\") pod \"aws-ebs-csi-driver-node-s9m6d\" (UID: \"fa48de0c-c971-487e-8bb8-c5e500ea2a33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.428497 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428443    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-multus-daemon-config\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.428497 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428465    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-dbus\") pod \"global-pull-secret-syncer-qw7gj\" (UID: \"6bbf80a3-a5e2-49a5-ad56-a21747cc4a16\") " pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:32.429038 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428507    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c6e1d829-065c-4516-a1cf-456e7d3ff8c1-agent-certs\") pod \"konnectivity-agent-bzhk8\" (UID: \"c6e1d829-065c-4516-a1cf-456e7d3ff8c1\") " pod="kube-system/konnectivity-agent-bzhk8"
Jul 28 22:04:32.429038 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428538    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-etc-modprobe-d\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.429038 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428589    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-run-openvswitch\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.429038 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428617    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-system-cni-dir\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.429038 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428656    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-sys\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.429038 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428692    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-run-ovn\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.429038 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428713    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-host-var-lib-cni-multus\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.429038 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428722    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-etc-modprobe-d\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.429038 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428738    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs\") pod \"network-metrics-daemon-dvqcn\" (UID: \"f738f750-e0f4-49e7-b7e6-0c151e664ce9\") " pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:32.429038 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428759    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9hvr\" (UniqueName: \"kubernetes.io/projected/f738f750-e0f4-49e7-b7e6-0c151e664ce9-kube-api-access-q9hvr\") pod \"network-metrics-daemon-dvqcn\" (UID: \"f738f750-e0f4-49e7-b7e6-0c151e664ce9\") " pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:32.429038 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428780    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c6e1d829-065c-4516-a1cf-456e7d3ff8c1-konnectivity-ca\") pod \"konnectivity-agent-bzhk8\" (UID: \"c6e1d829-065c-4516-a1cf-456e7d3ff8c1\") " pod="kube-system/konnectivity-agent-bzhk8"
Jul 28 22:04:32.429038 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428805    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-host-run-ovn-kubernetes\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.429038 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428868    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-multus-cni-dir\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.429038 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428893    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-host-run-multus-certs\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.429038 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428926    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-kubelet-config\") pod \"global-pull-secret-syncer-qw7gj\" (UID: \"6bbf80a3-a5e2-49a5-ad56-a21747cc4a16\") " pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:32.429038 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428948    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-cnibin\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.429038 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428971    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-etc-systemd\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.429842 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428948    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-sys\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.429842 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.428993    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dh6q\" (UniqueName: \"kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q\") pod \"network-check-target-np5j9\" (UID: \"8497aa3c-a69a-4e64-a1f3-306cd512ae81\") " pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:04:32.429842 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429014    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-log-socket\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.429842 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429033    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/435f03fc-de40-43d5-a9c9-7f22fb70825a-ovnkube-config\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.429842 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429051    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/435f03fc-de40-43d5-a9c9-7f22fb70825a-ovn-node-metrics-cert\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.429842 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429052    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-etc-systemd\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.429842 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429072    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-cni-binary-copy\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.429842 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429114    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzt24\" (UniqueName: \"kubernetes.io/projected/da509f2e-8338-4ece-9b46-ef348ba52ebc-kube-api-access-vzt24\") pod \"node-resolver-swmbg\" (UID: \"da509f2e-8338-4ece-9b46-ef348ba52ebc\") " pod="openshift-dns/node-resolver-swmbg"
Jul 28 22:04:32.429842 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429145    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-etc-sysconfig\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.429842 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429187    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dzjd\" (UniqueName: \"kubernetes.io/projected/d7a52716-3aa2-425f-bd5b-40184621f385-kube-api-access-4dzjd\") pod \"iptables-alerter-26gxp\" (UID: \"d7a52716-3aa2-425f-bd5b-40184621f385\") " pod="openshift-network-operator/iptables-alerter-26gxp"
Jul 28 22:04:32.429842 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429191    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-etc-sysconfig\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.429842 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429247    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-host-run-netns\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.429842 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429276    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fa48de0c-c971-487e-8bb8-c5e500ea2a33-sys-fs\") pod \"aws-ebs-csi-driver-node-s9m6d\" (UID: \"fa48de0c-c971-487e-8bb8-c5e500ea2a33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.429842 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429298    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/da509f2e-8338-4ece-9b46-ef348ba52ebc-hosts-file\") pod \"node-resolver-swmbg\" (UID: \"da509f2e-8338-4ece-9b46-ef348ba52ebc\") " pod="openshift-dns/node-resolver-swmbg"
Jul 28 22:04:32.429842 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429346    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.429842 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429383    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-var-lib-openvswitch\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.430642 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429402    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da509f2e-8338-4ece-9b46-ef348ba52ebc-tmp-dir\") pod \"node-resolver-swmbg\" (UID: \"da509f2e-8338-4ece-9b46-ef348ba52ebc\") " pod="openshift-dns/node-resolver-swmbg"
Jul 28 22:04:32.430642 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429422    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-system-cni-dir\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.430642 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429447    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-host\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.430642 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429470    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0efef71b-3b20-4214-914d-5138f89c14fc-host\") pod \"node-ca-gnhk8\" (UID: \"0efef71b-3b20-4214-914d-5138f89c14fc\") " pod="openshift-image-registry/node-ca-gnhk8"
Jul 28 22:04:32.430642 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429490    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-run-systemd\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.430642 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429527    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-node-log\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.430642 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429526    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-host\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.430642 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429558    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0efef71b-3b20-4214-914d-5138f89c14fc-host\") pod \"node-ca-gnhk8\" (UID: \"0efef71b-3b20-4214-914d-5138f89c14fc\") " pod="openshift-image-registry/node-ca-gnhk8"
Jul 28 22:04:32.430642 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429562    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-host-cni-bin\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.430642 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429557    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c6e1d829-065c-4516-a1cf-456e7d3ff8c1-konnectivity-ca\") pod \"konnectivity-agent-bzhk8\" (UID: \"c6e1d829-065c-4516-a1cf-456e7d3ff8c1\") " pod="kube-system/konnectivity-agent-bzhk8"
Jul 28 22:04:32.430642 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429616    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fa48de0c-c971-487e-8bb8-c5e500ea2a33-socket-dir\") pod \"aws-ebs-csi-driver-node-s9m6d\" (UID: \"fa48de0c-c971-487e-8bb8-c5e500ea2a33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.430642 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429676    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-multus-socket-dir-parent\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.430642 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429717    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-host-var-lib-cni-bin\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.430642 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429736    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-var-lib-kubelet\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.430642 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429761    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcl4k\" (UniqueName: \"kubernetes.io/projected/0efef71b-3b20-4214-914d-5138f89c14fc-kube-api-access-lcl4k\") pod \"node-ca-gnhk8\" (UID: \"0efef71b-3b20-4214-914d-5138f89c14fc\") " pod="openshift-image-registry/node-ca-gnhk8"
Jul 28 22:04:32.430642 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429784    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-cnibin\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.430642 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429805    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-multus-conf-dir\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.431448 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429819    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-var-lib-kubelet\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.431448 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429827    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-etc-kubernetes\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.431448 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429849    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret\") pod \"global-pull-secret-syncer-qw7gj\" (UID: \"6bbf80a3-a5e2-49a5-ad56-a21747cc4a16\") " pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:32.431448 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429875    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gt97\" (UniqueName: \"kubernetes.io/projected/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-kube-api-access-7gt97\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.431448 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429899    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b505b3f1-0db6-4d26-94c8-fdc9637e3415-tmp\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.431448 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429944    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-host-kubelet\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.431448 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429973    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/435f03fc-de40-43d5-a9c9-7f22fb70825a-env-overrides\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.431448 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.429995    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fa48de0c-c971-487e-8bb8-c5e500ea2a33-device-dir\") pod \"aws-ebs-csi-driver-node-s9m6d\" (UID: \"fa48de0c-c971-487e-8bb8-c5e500ea2a33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.431448 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.430019    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx76f\" (UniqueName: \"kubernetes.io/projected/fa48de0c-c971-487e-8bb8-c5e500ea2a33-kube-api-access-rx76f\") pod \"aws-ebs-csi-driver-node-s9m6d\" (UID: \"fa48de0c-c971-487e-8bb8-c5e500ea2a33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.431448 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.430062    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-systemd-units\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.431448 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.430096    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/435f03fc-de40-43d5-a9c9-7f22fb70825a-ovnkube-script-lib\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.431448 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.430122    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-cni-binary-copy\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.431448 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.430145    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-host-run-k8s-cni-cncf-io\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.431448 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.430168    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-host-run-netns\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.431448 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.430211    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-os-release\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.431448 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.430287    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-os-release\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.431448 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.430329    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-hostroot\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.432051 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.430352    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.432051 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.430381    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-etc-sysctl-conf\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.432051 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.430406    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0efef71b-3b20-4214-914d-5138f89c14fc-serviceca\") pod \"node-ca-gnhk8\" (UID: \"0efef71b-3b20-4214-914d-5138f89c14fc\") " pod="openshift-image-registry/node-ca-gnhk8"
Jul 28 22:04:32.432051 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.430444    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d7a52716-3aa2-425f-bd5b-40184621f385-iptables-alerter-script\") pod \"iptables-alerter-26gxp\" (UID: \"d7a52716-3aa2-425f-bd5b-40184621f385\") " pod="openshift-network-operator/iptables-alerter-26gxp"
Jul 28 22:04:32.432051 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.430528    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b505b3f1-0db6-4d26-94c8-fdc9637e3415-etc-sysctl-conf\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.432051 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.430525    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-host-cni-netd\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.432051 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.430571    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fa48de0c-c971-487e-8bb8-c5e500ea2a33-etc-selinux\") pod \"aws-ebs-csi-driver-node-s9m6d\" (UID: \"fa48de0c-c971-487e-8bb8-c5e500ea2a33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.432051 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.430934    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0efef71b-3b20-4214-914d-5138f89c14fc-serviceca\") pod \"node-ca-gnhk8\" (UID: \"0efef71b-3b20-4214-914d-5138f89c14fc\") " pod="openshift-image-registry/node-ca-gnhk8"
Jul 28 22:04:32.432051 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.431689    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b505b3f1-0db6-4d26-94c8-fdc9637e3415-etc-tuned\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.432051 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.431966    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c6e1d829-065c-4516-a1cf-456e7d3ff8c1-agent-certs\") pod \"konnectivity-agent-bzhk8\" (UID: \"c6e1d829-065c-4516-a1cf-456e7d3ff8c1\") " pod="kube-system/konnectivity-agent-bzhk8"
Jul 28 22:04:32.432496 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.432478    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b505b3f1-0db6-4d26-94c8-fdc9637e3415-tmp\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.438752 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.438735    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr848\" (UniqueName: \"kubernetes.io/projected/b505b3f1-0db6-4d26-94c8-fdc9637e3415-kube-api-access-tr848\") pod \"tuned-97vh9\" (UID: \"b505b3f1-0db6-4d26-94c8-fdc9637e3415\") " pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.439342 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.439326    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcl4k\" (UniqueName: \"kubernetes.io/projected/0efef71b-3b20-4214-914d-5138f89c14fc-kube-api-access-lcl4k\") pod \"node-ca-gnhk8\" (UID: \"0efef71b-3b20-4214-914d-5138f89c14fc\") " pod="openshift-image-registry/node-ca-gnhk8"
Jul 28 22:04:32.531250 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531208    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzt24\" (UniqueName: \"kubernetes.io/projected/da509f2e-8338-4ece-9b46-ef348ba52ebc-kube-api-access-vzt24\") pod \"node-resolver-swmbg\" (UID: \"da509f2e-8338-4ece-9b46-ef348ba52ebc\") " pod="openshift-dns/node-resolver-swmbg"
Jul 28 22:04:32.531445 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531259    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dzjd\" (UniqueName: \"kubernetes.io/projected/d7a52716-3aa2-425f-bd5b-40184621f385-kube-api-access-4dzjd\") pod \"iptables-alerter-26gxp\" (UID: \"d7a52716-3aa2-425f-bd5b-40184621f385\") " pod="openshift-network-operator/iptables-alerter-26gxp"
Jul 28 22:04:32.531445 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531287    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-host-run-netns\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.531445 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531310    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fa48de0c-c971-487e-8bb8-c5e500ea2a33-sys-fs\") pod \"aws-ebs-csi-driver-node-s9m6d\" (UID: \"fa48de0c-c971-487e-8bb8-c5e500ea2a33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.531445 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531327    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/da509f2e-8338-4ece-9b46-ef348ba52ebc-hosts-file\") pod \"node-resolver-swmbg\" (UID: \"da509f2e-8338-4ece-9b46-ef348ba52ebc\") " pod="openshift-dns/node-resolver-swmbg"
Jul 28 22:04:32.531445 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531395    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/da509f2e-8338-4ece-9b46-ef348ba52ebc-hosts-file\") pod \"node-resolver-swmbg\" (UID: \"da509f2e-8338-4ece-9b46-ef348ba52ebc\") " pod="openshift-dns/node-resolver-swmbg"
Jul 28 22:04:32.531445 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531394    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fa48de0c-c971-487e-8bb8-c5e500ea2a33-sys-fs\") pod \"aws-ebs-csi-driver-node-s9m6d\" (UID: \"fa48de0c-c971-487e-8bb8-c5e500ea2a33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.531445 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531393    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.531445 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531445    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-host-run-netns\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.531836 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531457    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-var-lib-openvswitch\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.531836 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531480    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da509f2e-8338-4ece-9b46-ef348ba52ebc-tmp-dir\") pod \"node-resolver-swmbg\" (UID: \"da509f2e-8338-4ece-9b46-ef348ba52ebc\") " pod="openshift-dns/node-resolver-swmbg"
Jul 28 22:04:32.531836 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531507    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-system-cni-dir\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.531836 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531546    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-var-lib-openvswitch\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.531836 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531553    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-system-cni-dir\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.531836 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531581    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-run-systemd\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.531836 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531612    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-node-log\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.531836 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531642    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-run-systemd\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.531836 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531674    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-node-log\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.531836 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531682    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-host-cni-bin\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.531836 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531647    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-host-cni-bin\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.531836 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531708    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fa48de0c-c971-487e-8bb8-c5e500ea2a33-socket-dir\") pod \"aws-ebs-csi-driver-node-s9m6d\" (UID: \"fa48de0c-c971-487e-8bb8-c5e500ea2a33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.531836 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531732    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-multus-socket-dir-parent\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.531836 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531739    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da509f2e-8338-4ece-9b46-ef348ba52ebc-tmp-dir\") pod \"node-resolver-swmbg\" (UID: \"da509f2e-8338-4ece-9b46-ef348ba52ebc\") " pod="openshift-dns/node-resolver-swmbg"
Jul 28 22:04:32.531836 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531756    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-host-var-lib-cni-bin\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.531836 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531780    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-cnibin\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.531836 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531805    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-host-var-lib-cni-bin\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.531836 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531804    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-multus-conf-dir\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.532607 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531826    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-etc-kubernetes\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.532607 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531827    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fa48de0c-c971-487e-8bb8-c5e500ea2a33-socket-dir\") pod \"aws-ebs-csi-driver-node-s9m6d\" (UID: \"fa48de0c-c971-487e-8bb8-c5e500ea2a33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.532607 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531834    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-multus-conf-dir\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.532607 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531782    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-multus-socket-dir-parent\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.532607 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531860    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-etc-kubernetes\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.532607 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531860    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret\") pod \"global-pull-secret-syncer-qw7gj\" (UID: \"6bbf80a3-a5e2-49a5-ad56-a21747cc4a16\") " pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:32.532607 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531899    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gt97\" (UniqueName: \"kubernetes.io/projected/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-kube-api-access-7gt97\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.532607 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531934    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-host-kubelet\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.532607 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531947    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-cnibin\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.532607 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:32.531951    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:32.532607 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531962    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/435f03fc-de40-43d5-a9c9-7f22fb70825a-env-overrides\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.532607 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531947    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.532607 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531987    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fa48de0c-c971-487e-8bb8-c5e500ea2a33-device-dir\") pod \"aws-ebs-csi-driver-node-s9m6d\" (UID: \"fa48de0c-c971-487e-8bb8-c5e500ea2a33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.532607 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.531993    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-host-kubelet\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.532607 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:32.532356    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret podName:6bbf80a3-a5e2-49a5-ad56-a21747cc4a16 nodeName:}" failed. No retries permitted until 2026-07-28 22:04:33.032001424 +0000 UTC m=+2.016144038 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret") pod "global-pull-secret-syncer-qw7gj" (UID: "6bbf80a3-a5e2-49a5-ad56-a21747cc4a16") : object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:32.532607 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.532369    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fa48de0c-c971-487e-8bb8-c5e500ea2a33-device-dir\") pod \"aws-ebs-csi-driver-node-s9m6d\" (UID: \"fa48de0c-c971-487e-8bb8-c5e500ea2a33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.532607 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.532529    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx76f\" (UniqueName: \"kubernetes.io/projected/fa48de0c-c971-487e-8bb8-c5e500ea2a33-kube-api-access-rx76f\") pod \"aws-ebs-csi-driver-node-s9m6d\" (UID: \"fa48de0c-c971-487e-8bb8-c5e500ea2a33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.533412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.532570    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-systemd-units\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.533412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.532603    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/435f03fc-de40-43d5-a9c9-7f22fb70825a-ovnkube-script-lib\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.533412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.532631    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-cni-binary-copy\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.533412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.532664    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-host-run-k8s-cni-cncf-io\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.533412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.532695    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-host-run-netns\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.533412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.532725    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-os-release\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.533412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.532793    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-os-release\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.533412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.532804    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-os-release\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.533412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.532836    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-hostroot\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.533412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.532844    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-host-run-netns\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.533412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.532849    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-host-run-k8s-cni-cncf-io\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.533412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.532901    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-os-release\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.533412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.532947    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.533412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.532973    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-systemd-units\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.533412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.532955    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/435f03fc-de40-43d5-a9c9-7f22fb70825a-env-overrides\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.533412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.532987    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d7a52716-3aa2-425f-bd5b-40184621f385-iptables-alerter-script\") pod \"iptables-alerter-26gxp\" (UID: \"d7a52716-3aa2-425f-bd5b-40184621f385\") " pod="openshift-network-operator/iptables-alerter-26gxp"
Jul 28 22:04:32.533412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533053    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-host-cni-netd\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.533412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533081    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fa48de0c-c971-487e-8bb8-c5e500ea2a33-etc-selinux\") pod \"aws-ebs-csi-driver-node-s9m6d\" (UID: \"fa48de0c-c971-487e-8bb8-c5e500ea2a33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.534207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533113    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-etc-openvswitch\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.534207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533145    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.534207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533167    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.534207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533184    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.534207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533217    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-host-var-lib-kubelet\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.534207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533261    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg85w\" (UniqueName: \"kubernetes.io/projected/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-kube-api-access-kg85w\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.534207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533294    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7a52716-3aa2-425f-bd5b-40184621f385-host-slash\") pod \"iptables-alerter-26gxp\" (UID: \"d7a52716-3aa2-425f-bd5b-40184621f385\") " pod="openshift-network-operator/iptables-alerter-26gxp"
Jul 28 22:04:32.534207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533323    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-host-slash\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.534207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533321    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-cni-binary-copy\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.534207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533347    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fa48de0c-c971-487e-8bb8-c5e500ea2a33-etc-selinux\") pod \"aws-ebs-csi-driver-node-s9m6d\" (UID: \"fa48de0c-c971-487e-8bb8-c5e500ea2a33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.534207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533355    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62w85\" (UniqueName: \"kubernetes.io/projected/435f03fc-de40-43d5-a9c9-7f22fb70825a-kube-api-access-62w85\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.534207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533474    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-host-var-lib-kubelet\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.534207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533466    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-hostroot\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.534207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533499    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fa48de0c-c971-487e-8bb8-c5e500ea2a33-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s9m6d\" (UID: \"fa48de0c-c971-487e-8bb8-c5e500ea2a33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.534207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533545    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fa48de0c-c971-487e-8bb8-c5e500ea2a33-registration-dir\") pod \"aws-ebs-csi-driver-node-s9m6d\" (UID: \"fa48de0c-c971-487e-8bb8-c5e500ea2a33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.534207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533586    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-multus-daemon-config\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.534207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533620    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-dbus\") pod \"global-pull-secret-syncer-qw7gj\" (UID: \"6bbf80a3-a5e2-49a5-ad56-a21747cc4a16\") " pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:32.534949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533661    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-run-openvswitch\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.534949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533694    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-system-cni-dir\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.534949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533725    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-run-ovn\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.534949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533758    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-host-var-lib-cni-multus\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.534949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533787    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs\") pod \"network-metrics-daemon-dvqcn\" (UID: \"f738f750-e0f4-49e7-b7e6-0c151e664ce9\") " pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:32.534949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533820    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9hvr\" (UniqueName: \"kubernetes.io/projected/f738f750-e0f4-49e7-b7e6-0c151e664ce9-kube-api-access-q9hvr\") pod \"network-metrics-daemon-dvqcn\" (UID: \"f738f750-e0f4-49e7-b7e6-0c151e664ce9\") " pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:32.534949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533861    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-host-run-ovn-kubernetes\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.534949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533898    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-multus-cni-dir\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.534949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.533974    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-host-run-multus-certs\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.534949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.534004    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-kubelet-config\") pod \"global-pull-secret-syncer-qw7gj\" (UID: \"6bbf80a3-a5e2-49a5-ad56-a21747cc4a16\") " pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:32.534949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.534018    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d7a52716-3aa2-425f-bd5b-40184621f385-iptables-alerter-script\") pod \"iptables-alerter-26gxp\" (UID: \"d7a52716-3aa2-425f-bd5b-40184621f385\") " pod="openshift-network-operator/iptables-alerter-26gxp"
Jul 28 22:04:32.534949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.534037    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-cnibin\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.534949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.534074    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dh6q\" (UniqueName: \"kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q\") pod \"network-check-target-np5j9\" (UID: \"8497aa3c-a69a-4e64-a1f3-306cd512ae81\") " pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:04:32.534949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.534098    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.534949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.534107    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-log-socket\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.534949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.534140    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/435f03fc-de40-43d5-a9c9-7f22fb70825a-ovnkube-config\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.534949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.534151    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/435f03fc-de40-43d5-a9c9-7f22fb70825a-ovnkube-script-lib\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.535762 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.534168    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/435f03fc-de40-43d5-a9c9-7f22fb70825a-ovn-node-metrics-cert\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.535762 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.534226    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-run-openvswitch\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.535762 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.534285    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-cni-binary-copy\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.535762 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.534102    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-run-ovn\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.535762 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.534442    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.535762 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.534506    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-kubelet-config\") pod \"global-pull-secret-syncer-qw7gj\" (UID: \"6bbf80a3-a5e2-49a5-ad56-a21747cc4a16\") " pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:32.535762 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.534539    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-dbus\") pod \"global-pull-secret-syncer-qw7gj\" (UID: \"6bbf80a3-a5e2-49a5-ad56-a21747cc4a16\") " pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:32.535762 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.534561    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-host-var-lib-cni-multus\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.535762 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.534584    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-host-slash\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.535762 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.534674    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-host-run-multus-certs\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.535762 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.534679    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-multus-daemon-config\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.535762 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.534750    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-etc-openvswitch\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.535762 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.534817    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-system-cni-dir\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.535762 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.534996    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fa48de0c-c971-487e-8bb8-c5e500ea2a33-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s9m6d\" (UID: \"fa48de0c-c971-487e-8bb8-c5e500ea2a33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.535762 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.535077    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-cni-binary-copy\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.535762 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.535084    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-cnibin\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.535762 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.535154    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-host-cni-netd\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.536545 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.535165    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7a52716-3aa2-425f-bd5b-40184621f385-host-slash\") pod \"iptables-alerter-26gxp\" (UID: \"d7a52716-3aa2-425f-bd5b-40184621f385\") " pod="openshift-network-operator/iptables-alerter-26gxp"
Jul 28 22:04:32.536545 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.535219    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-log-socket\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.536545 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.535219    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-multus-cni-dir\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.536545 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.535305    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fa48de0c-c971-487e-8bb8-c5e500ea2a33-registration-dir\") pod \"aws-ebs-csi-driver-node-s9m6d\" (UID: \"fa48de0c-c971-487e-8bb8-c5e500ea2a33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.536545 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:32.535343    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:32.536545 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:32.535410    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs podName:f738f750-e0f4-49e7-b7e6-0c151e664ce9 nodeName:}" failed. No retries permitted until 2026-07-28 22:04:33.035392559 +0000 UTC m=+2.019535172 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs") pod "network-metrics-daemon-dvqcn" (UID: "f738f750-e0f4-49e7-b7e6-0c151e664ce9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:32.536545 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.535410    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/435f03fc-de40-43d5-a9c9-7f22fb70825a-host-run-ovn-kubernetes\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.536545 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.535787    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/435f03fc-de40-43d5-a9c9-7f22fb70825a-ovnkube-config\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.537190 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.537173    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/435f03fc-de40-43d5-a9c9-7f22fb70825a-ovn-node-metrics-cert\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.541107 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:32.541078    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 22:04:32.541107 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:32.541107    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 22:04:32.541292 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:32.541121    2574 projected.go:196] Error preparing data for projected volume kube-api-access-6dh6q for pod openshift-network-diagnostics/network-check-target-np5j9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:32.541292 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:32.541184    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q podName:8497aa3c-a69a-4e64-a1f3-306cd512ae81 nodeName:}" failed. No retries permitted until 2026-07-28 22:04:33.041166091 +0000 UTC m=+2.025308706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6dh6q" (UniqueName: "kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q") pod "network-check-target-np5j9" (UID: "8497aa3c-a69a-4e64-a1f3-306cd512ae81") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:32.541672 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.541623    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gt97\" (UniqueName: \"kubernetes.io/projected/1d63809c-6d3f-4a78-9c1c-24b2cf6223f4-kube-api-access-7gt97\") pod \"multus-additional-cni-plugins-mj477\" (UID: \"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4\") " pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.542245 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.541894    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzt24\" (UniqueName: \"kubernetes.io/projected/da509f2e-8338-4ece-9b46-ef348ba52ebc-kube-api-access-vzt24\") pod \"node-resolver-swmbg\" (UID: \"da509f2e-8338-4ece-9b46-ef348ba52ebc\") " pod="openshift-dns/node-resolver-swmbg"
Jul 28 22:04:32.542346 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.541905    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg85w\" (UniqueName: \"kubernetes.io/projected/c7cde400-9ef6-4b09-9cc5-f1d05d424e6c-kube-api-access-kg85w\") pod \"multus-5vhbh\" (UID: \"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c\") " pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.542346 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.542176    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dzjd\" (UniqueName: \"kubernetes.io/projected/d7a52716-3aa2-425f-bd5b-40184621f385-kube-api-access-4dzjd\") pod \"iptables-alerter-26gxp\" (UID: \"d7a52716-3aa2-425f-bd5b-40184621f385\") " pod="openshift-network-operator/iptables-alerter-26gxp"
Jul 28 22:04:32.542346 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.542184    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx76f\" (UniqueName: \"kubernetes.io/projected/fa48de0c-c971-487e-8bb8-c5e500ea2a33-kube-api-access-rx76f\") pod \"aws-ebs-csi-driver-node-s9m6d\" (UID: \"fa48de0c-c971-487e-8bb8-c5e500ea2a33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.543164 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.543143    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62w85\" (UniqueName: \"kubernetes.io/projected/435f03fc-de40-43d5-a9c9-7f22fb70825a-kube-api-access-62w85\") pod \"ovnkube-node-jrh49\" (UID: \"435f03fc-de40-43d5-a9c9-7f22fb70825a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.543317 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.543224    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9hvr\" (UniqueName: \"kubernetes.io/projected/f738f750-e0f4-49e7-b7e6-0c151e664ce9-kube-api-access-q9hvr\") pod \"network-metrics-daemon-dvqcn\" (UID: \"f738f750-e0f4-49e7-b7e6-0c151e664ce9\") " pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:32.571650 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:32.571620    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54f8ab6a0dce2d047f0664fad06551cb.slice/crio-9b3e3c0385f90f15e62e0bfc6116fe9288a80daf4b0161080bcf833694d9f444 WatchSource:0}: Error finding container 9b3e3c0385f90f15e62e0bfc6116fe9288a80daf4b0161080bcf833694d9f444: Status 404 returned error can't find the container with id 9b3e3c0385f90f15e62e0bfc6116fe9288a80daf4b0161080bcf833694d9f444
Jul 28 22:04:32.571970 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:32.571955    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cc3a9f08dccc44aae468666893de9e2.slice/crio-19afb167cea8b8c02afebcdfd231ebd24b3b94e56ac9e1e9bfdb81dfcda37b1b WatchSource:0}: Error finding container 19afb167cea8b8c02afebcdfd231ebd24b3b94e56ac9e1e9bfdb81dfcda37b1b: Status 404 returned error can't find the container with id 19afb167cea8b8c02afebcdfd231ebd24b3b94e56ac9e1e9bfdb81dfcda37b1b
Jul 28 22:04:32.576543 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.576519    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 22:04:32.645277 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.645184    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bzhk8"
Jul 28 22:04:32.650336 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.650320    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-97vh9"
Jul 28 22:04:32.655222 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:32.655196    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6e1d829_065c_4516_a1cf_456e7d3ff8c1.slice/crio-178350f7106d11ddeb7a4cbcbb2fe769a878093d09bd260b15bcb0135f908933 WatchSource:0}: Error finding container 178350f7106d11ddeb7a4cbcbb2fe769a878093d09bd260b15bcb0135f908933: Status 404 returned error can't find the container with id 178350f7106d11ddeb7a4cbcbb2fe769a878093d09bd260b15bcb0135f908933
Jul 28 22:04:32.659063 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:32.659045    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb505b3f1_0db6_4d26_94c8_fdc9637e3415.slice/crio-d9dbf6d6beeaf4671634c04c6bdc81b830ed6996efa0bd6a445a68db8bd9342c WatchSource:0}: Error finding container d9dbf6d6beeaf4671634c04c6bdc81b830ed6996efa0bd6a445a68db8bd9342c: Status 404 returned error can't find the container with id d9dbf6d6beeaf4671634c04c6bdc81b830ed6996efa0bd6a445a68db8bd9342c
Jul 28 22:04:32.666334 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.666314    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gnhk8"
Jul 28 22:04:32.675465 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:32.675441    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0efef71b_3b20_4214_914d_5138f89c14fc.slice/crio-d78870e2b7f6e3917e97d22cea78dff0ff212af226bb64a698afd8e81f0a710a WatchSource:0}: Error finding container d78870e2b7f6e3917e97d22cea78dff0ff212af226bb64a698afd8e81f0a710a: Status 404 returned error can't find the container with id d78870e2b7f6e3917e97d22cea78dff0ff212af226bb64a698afd8e81f0a710a
Jul 28 22:04:32.679521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.679502    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5vhbh"
Jul 28 22:04:32.687556 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.687527    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-26gxp"
Jul 28 22:04:32.688187 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:32.688161    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7cde400_9ef6_4b09_9cc5_f1d05d424e6c.slice/crio-8486360d04d110efeff9eeda286bf775dc5b761785247a54978d6b826ffdc677 WatchSource:0}: Error finding container 8486360d04d110efeff9eeda286bf775dc5b761785247a54978d6b826ffdc677: Status 404 returned error can't find the container with id 8486360d04d110efeff9eeda286bf775dc5b761785247a54978d6b826ffdc677
Jul 28 22:04:32.695541 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:32.695521    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7a52716_3aa2_425f_bd5b_40184621f385.slice/crio-a7239dcc2ca83cdc4484a047f6925a3579ab37fa24a5b59df1bd3871c4a93e23 WatchSource:0}: Error finding container a7239dcc2ca83cdc4484a047f6925a3579ab37fa24a5b59df1bd3871c4a93e23: Status 404 returned error can't find the container with id a7239dcc2ca83cdc4484a047f6925a3579ab37fa24a5b59df1bd3871c4a93e23
Jul 28 22:04:32.710871 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.710842    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:32.719085 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:32.719059    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod435f03fc_de40_43d5_a9c9_7f22fb70825a.slice/crio-47cfd17631aa87a02b2f57d71300a6a862216112075125c948d002d7b4110c21 WatchSource:0}: Error finding container 47cfd17631aa87a02b2f57d71300a6a862216112075125c948d002d7b4110c21: Status 404 returned error can't find the container with id 47cfd17631aa87a02b2f57d71300a6a862216112075125c948d002d7b4110c21
Jul 28 22:04:32.728910 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.728893    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d"
Jul 28 22:04:32.736598 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:32.736572    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa48de0c_c971_487e_8bb8_c5e500ea2a33.slice/crio-3db3e1d9b362f705ff39fdfdb6208989fad88a59e253fc77d3ea797e605967bd WatchSource:0}: Error finding container 3db3e1d9b362f705ff39fdfdb6208989fad88a59e253fc77d3ea797e605967bd: Status 404 returned error can't find the container with id 3db3e1d9b362f705ff39fdfdb6208989fad88a59e253fc77d3ea797e605967bd
Jul 28 22:04:32.737757 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.737741    2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 22:04:32.749693 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.749674    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-swmbg"
Jul 28 22:04:32.750832 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.750815    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mj477"
Jul 28 22:04:32.758397 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:32.758376    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda509f2e_8338_4ece_9b46_ef348ba52ebc.slice/crio-f35e61d24a98825a722f6009c98b4372f27aa2d895703e2ad6423ce760a43e3e WatchSource:0}: Error finding container f35e61d24a98825a722f6009c98b4372f27aa2d895703e2ad6423ce760a43e3e: Status 404 returned error can't find the container with id f35e61d24a98825a722f6009c98b4372f27aa2d895703e2ad6423ce760a43e3e
Jul 28 22:04:32.760455 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:04:32.760434    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d63809c_6d3f_4a78_9c1c_24b2cf6223f4.slice/crio-3c87fc6073731ae3f22afd1f24562b37ca4e709f60186123e5dbb66b6b8bea7c WatchSource:0}: Error finding container 3c87fc6073731ae3f22afd1f24562b37ca4e709f60186123e5dbb66b6b8bea7c: Status 404 returned error can't find the container with id 3c87fc6073731ae3f22afd1f24562b37ca4e709f60186123e5dbb66b6b8bea7c
Jul 28 22:04:32.762088 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:32.762071    2574 kuberuntime_manager.go:1459] "Unhandled Error" err="init container egress-router-binary-copy start failed in pod multus-additional-cni-plugins-mj477_openshift-multus(1d63809c-6d3f-4a78-9c1c-24b2cf6223f4): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 28 22:04:32.762140 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:32.762102    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"egress-router-binary-copy\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-multus/multus-additional-cni-plugins-mj477" podUID="1d63809c-6d3f-4a78-9c1c-24b2cf6223f4"
Jul 28 22:04:32.899584 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:32.899504    2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 22:04:33.038009 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:33.037974    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs\") pod \"network-metrics-daemon-dvqcn\" (UID: \"f738f750-e0f4-49e7-b7e6-0c151e664ce9\") " pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:33.038185 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:33.038033    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret\") pod \"global-pull-secret-syncer-qw7gj\" (UID: \"6bbf80a3-a5e2-49a5-ad56-a21747cc4a16\") " pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:33.038185 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:33.038128    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:33.038326 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:33.038192    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret podName:6bbf80a3-a5e2-49a5-ad56-a21747cc4a16 nodeName:}" failed. No retries permitted until 2026-07-28 22:04:34.038172869 +0000 UTC m=+3.022315472 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret") pod "global-pull-secret-syncer-qw7gj" (UID: "6bbf80a3-a5e2-49a5-ad56-a21747cc4a16") : object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:33.038326 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:33.038129    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:33.038326 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:33.038298    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs podName:f738f750-e0f4-49e7-b7e6-0c151e664ce9 nodeName:}" failed. No retries permitted until 2026-07-28 22:04:34.03828129 +0000 UTC m=+3.022423895 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs") pod "network-metrics-daemon-dvqcn" (UID: "f738f750-e0f4-49e7-b7e6-0c151e664ce9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:33.138461 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:33.138418    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dh6q\" (UniqueName: \"kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q\") pod \"network-check-target-np5j9\" (UID: \"8497aa3c-a69a-4e64-a1f3-306cd512ae81\") " pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:04:33.138642 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:33.138610    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 22:04:33.138642 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:33.138639    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 22:04:33.138812 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:33.138652    2574 projected.go:196] Error preparing data for projected volume kube-api-access-6dh6q for pod openshift-network-diagnostics/network-check-target-np5j9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:33.138812 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:33.138705    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q podName:8497aa3c-a69a-4e64-a1f3-306cd512ae81 nodeName:}" failed. No retries permitted until 2026-07-28 22:04:34.138688297 +0000 UTC m=+3.122830910 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6dh6q" (UniqueName: "kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q") pod "network-check-target-np5j9" (UID: "8497aa3c-a69a-4e64-a1f3-306cd512ae81") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:33.367871 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:33.367435    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 21:59:32 +0000 UTC" deadline="2027-12-31 14:03:40.395297574 +0000 UTC"
Jul 28 22:04:33.367871 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:33.367618    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12495h59m7.027688197s"
Jul 28 22:04:33.488919 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:33.488369    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:33.488919 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:33.488515    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvqcn" podUID="f738f750-e0f4-49e7-b7e6-0c151e664ce9"
Jul 28 22:04:33.502873 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:33.502811    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mj477" event={"ID":"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4","Type":"ContainerStarted","Data":"3c87fc6073731ae3f22afd1f24562b37ca4e709f60186123e5dbb66b6b8bea7c"}
Jul 28 22:04:33.507723 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:33.507686    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"egress-router-binary-copy\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-multus/multus-additional-cni-plugins-mj477" podUID="1d63809c-6d3f-4a78-9c1c-24b2cf6223f4"
Jul 28 22:04:33.511991 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:33.511963    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jrh49" event={"ID":"435f03fc-de40-43d5-a9c9-7f22fb70825a","Type":"ContainerStarted","Data":"47cfd17631aa87a02b2f57d71300a6a862216112075125c948d002d7b4110c21"}
Jul 28 22:04:33.514256 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:33.514213    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-26gxp" event={"ID":"d7a52716-3aa2-425f-bd5b-40184621f385","Type":"ContainerStarted","Data":"a7239dcc2ca83cdc4484a047f6925a3579ab37fa24a5b59df1bd3871c4a93e23"}
Jul 28 22:04:33.520243 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:33.520197    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-97vh9" event={"ID":"b505b3f1-0db6-4d26-94c8-fdc9637e3415","Type":"ContainerStarted","Data":"d9dbf6d6beeaf4671634c04c6bdc81b830ed6996efa0bd6a445a68db8bd9342c"}
Jul 28 22:04:33.538438 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:33.538333    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bzhk8" event={"ID":"c6e1d829-065c-4516-a1cf-456e7d3ff8c1","Type":"ContainerStarted","Data":"178350f7106d11ddeb7a4cbcbb2fe769a878093d09bd260b15bcb0135f908933"}
Jul 28 22:04:33.541045 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:33.541008    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-69.ec2.internal" event={"ID":"54f8ab6a0dce2d047f0664fad06551cb","Type":"ContainerStarted","Data":"9b3e3c0385f90f15e62e0bfc6116fe9288a80daf4b0161080bcf833694d9f444"}
Jul 28 22:04:33.544461 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:33.544384    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-69.ec2.internal" event={"ID":"7cc3a9f08dccc44aae468666893de9e2","Type":"ContainerStarted","Data":"19afb167cea8b8c02afebcdfd231ebd24b3b94e56ac9e1e9bfdb81dfcda37b1b"}
Jul 28 22:04:33.548562 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:33.548449    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-swmbg" event={"ID":"da509f2e-8338-4ece-9b46-ef348ba52ebc","Type":"ContainerStarted","Data":"f35e61d24a98825a722f6009c98b4372f27aa2d895703e2ad6423ce760a43e3e"}
Jul 28 22:04:33.551995 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:33.551956    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d" event={"ID":"fa48de0c-c971-487e-8bb8-c5e500ea2a33","Type":"ContainerStarted","Data":"3db3e1d9b362f705ff39fdfdb6208989fad88a59e253fc77d3ea797e605967bd"}
Jul 28 22:04:33.562945 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:33.562914    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5vhbh" event={"ID":"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c","Type":"ContainerStarted","Data":"8486360d04d110efeff9eeda286bf775dc5b761785247a54978d6b826ffdc677"}
Jul 28 22:04:33.564929 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:33.564908    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gnhk8" event={"ID":"0efef71b-3b20-4214-914d-5138f89c14fc","Type":"ContainerStarted","Data":"d78870e2b7f6e3917e97d22cea78dff0ff212af226bb64a698afd8e81f0a710a"}
Jul 28 22:04:33.640098 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:33.640023    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 22:04:34.047204 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:34.046332    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret\") pod \"global-pull-secret-syncer-qw7gj\" (UID: \"6bbf80a3-a5e2-49a5-ad56-a21747cc4a16\") " pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:34.047204 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:34.046417    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs\") pod \"network-metrics-daemon-dvqcn\" (UID: \"f738f750-e0f4-49e7-b7e6-0c151e664ce9\") " pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:34.047204 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:34.046593    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:34.047204 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:34.046662    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs podName:f738f750-e0f4-49e7-b7e6-0c151e664ce9 nodeName:}" failed. No retries permitted until 2026-07-28 22:04:36.046642526 +0000 UTC m=+5.030785128 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs") pod "network-metrics-daemon-dvqcn" (UID: "f738f750-e0f4-49e7-b7e6-0c151e664ce9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:34.047204 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:34.047074    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:34.047204 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:34.047123    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret podName:6bbf80a3-a5e2-49a5-ad56-a21747cc4a16 nodeName:}" failed. No retries permitted until 2026-07-28 22:04:36.047107304 +0000 UTC m=+5.031249915 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret") pod "global-pull-secret-syncer-qw7gj" (UID: "6bbf80a3-a5e2-49a5-ad56-a21747cc4a16") : object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:34.147510 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:34.147470    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dh6q\" (UniqueName: \"kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q\") pod \"network-check-target-np5j9\" (UID: \"8497aa3c-a69a-4e64-a1f3-306cd512ae81\") " pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:04:34.147700 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:34.147661    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 22:04:34.147700 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:34.147684    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 22:04:34.147700 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:34.147697    2574 projected.go:196] Error preparing data for projected volume kube-api-access-6dh6q for pod openshift-network-diagnostics/network-check-target-np5j9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:34.147864 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:34.147755    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q podName:8497aa3c-a69a-4e64-a1f3-306cd512ae81 nodeName:}" failed. No retries permitted until 2026-07-28 22:04:36.147737108 +0000 UTC m=+5.131879708 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6dh6q" (UniqueName: "kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q") pod "network-check-target-np5j9" (UID: "8497aa3c-a69a-4e64-a1f3-306cd512ae81") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:34.345961 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:34.345883    2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 22:04:34.368051 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:34.367988    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 21:59:32 +0000 UTC" deadline="2028-03-28 00:32:11.130645629 +0000 UTC"
Jul 28 22:04:34.368051 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:34.368049    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14594h27m36.762600838s"
Jul 28 22:04:34.488461 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:34.487743    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:04:34.488461 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:34.487874    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-np5j9" podUID="8497aa3c-a69a-4e64-a1f3-306cd512ae81"
Jul 28 22:04:34.488461 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:34.488293    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:34.488461 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:34.488396    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw7gj" podUID="6bbf80a3-a5e2-49a5-ad56-a21747cc4a16"
Jul 28 22:04:35.489092 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:35.489035    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:35.489592 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:35.489181    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvqcn" podUID="f738f750-e0f4-49e7-b7e6-0c151e664ce9"
Jul 28 22:04:36.063440 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:36.063405    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret\") pod \"global-pull-secret-syncer-qw7gj\" (UID: \"6bbf80a3-a5e2-49a5-ad56-a21747cc4a16\") " pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:36.063624 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:36.063482    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs\") pod \"network-metrics-daemon-dvqcn\" (UID: \"f738f750-e0f4-49e7-b7e6-0c151e664ce9\") " pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:36.063706 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:36.063631    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:36.063706 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:36.063694    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs podName:f738f750-e0f4-49e7-b7e6-0c151e664ce9 nodeName:}" failed. No retries permitted until 2026-07-28 22:04:40.063674628 +0000 UTC m=+9.047817228 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs") pod "network-metrics-daemon-dvqcn" (UID: "f738f750-e0f4-49e7-b7e6-0c151e664ce9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:36.064196 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:36.064074    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:36.064196 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:36.064125    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret podName:6bbf80a3-a5e2-49a5-ad56-a21747cc4a16 nodeName:}" failed. No retries permitted until 2026-07-28 22:04:40.06411004 +0000 UTC m=+9.048252640 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret") pod "global-pull-secret-syncer-qw7gj" (UID: "6bbf80a3-a5e2-49a5-ad56-a21747cc4a16") : object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:36.164364 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:36.164326    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dh6q\" (UniqueName: \"kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q\") pod \"network-check-target-np5j9\" (UID: \"8497aa3c-a69a-4e64-a1f3-306cd512ae81\") " pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:04:36.164671 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:36.164632    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 22:04:36.164671 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:36.164653    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 22:04:36.164671 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:36.164669    2574 projected.go:196] Error preparing data for projected volume kube-api-access-6dh6q for pod openshift-network-diagnostics/network-check-target-np5j9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:36.165025 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:36.164729    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q podName:8497aa3c-a69a-4e64-a1f3-306cd512ae81 nodeName:}" failed. No retries permitted until 2026-07-28 22:04:40.164709963 +0000 UTC m=+9.148852575 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6dh6q" (UniqueName: "kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q") pod "network-check-target-np5j9" (UID: "8497aa3c-a69a-4e64-a1f3-306cd512ae81") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:36.488185 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:36.487901    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:36.488185 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:36.487946    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:04:36.488185 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:36.488043    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw7gj" podUID="6bbf80a3-a5e2-49a5-ad56-a21747cc4a16"
Jul 28 22:04:36.488512 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:36.488245    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-np5j9" podUID="8497aa3c-a69a-4e64-a1f3-306cd512ae81"
Jul 28 22:04:37.487959 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:37.487923    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:37.488450 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:37.488080    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvqcn" podUID="f738f750-e0f4-49e7-b7e6-0c151e664ce9"
Jul 28 22:04:38.487560 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:38.487530    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:38.487761 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:38.487569    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:04:38.487761 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:38.487665    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw7gj" podUID="6bbf80a3-a5e2-49a5-ad56-a21747cc4a16"
Jul 28 22:04:38.487877 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:38.487787    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-np5j9" podUID="8497aa3c-a69a-4e64-a1f3-306cd512ae81"
Jul 28 22:04:39.488408 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:39.488332    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:39.488778 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:39.488492    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvqcn" podUID="f738f750-e0f4-49e7-b7e6-0c151e664ce9"
Jul 28 22:04:40.097106 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:40.097064    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret\") pod \"global-pull-secret-syncer-qw7gj\" (UID: \"6bbf80a3-a5e2-49a5-ad56-a21747cc4a16\") " pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:40.097331 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:40.097140    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs\") pod \"network-metrics-daemon-dvqcn\" (UID: \"f738f750-e0f4-49e7-b7e6-0c151e664ce9\") " pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:40.097331 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:40.097244    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:40.097331 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:40.097262    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:40.097331 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:40.097310    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs podName:f738f750-e0f4-49e7-b7e6-0c151e664ce9 nodeName:}" failed. No retries permitted until 2026-07-28 22:04:48.097292242 +0000 UTC m=+17.081434841 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs") pod "network-metrics-daemon-dvqcn" (UID: "f738f750-e0f4-49e7-b7e6-0c151e664ce9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:40.097331 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:40.097326    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret podName:6bbf80a3-a5e2-49a5-ad56-a21747cc4a16 nodeName:}" failed. No retries permitted until 2026-07-28 22:04:48.097318123 +0000 UTC m=+17.081460720 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret") pod "global-pull-secret-syncer-qw7gj" (UID: "6bbf80a3-a5e2-49a5-ad56-a21747cc4a16") : object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:40.198360 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:40.198259    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dh6q\" (UniqueName: \"kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q\") pod \"network-check-target-np5j9\" (UID: \"8497aa3c-a69a-4e64-a1f3-306cd512ae81\") " pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:04:40.198545 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:40.198491    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 22:04:40.198608 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:40.198519    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 22:04:40.198608 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:40.198562    2574 projected.go:196] Error preparing data for projected volume kube-api-access-6dh6q for pod openshift-network-diagnostics/network-check-target-np5j9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:40.198721 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:40.198629    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q podName:8497aa3c-a69a-4e64-a1f3-306cd512ae81 nodeName:}" failed. No retries permitted until 2026-07-28 22:04:48.198610126 +0000 UTC m=+17.182752741 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6dh6q" (UniqueName: "kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q") pod "network-check-target-np5j9" (UID: "8497aa3c-a69a-4e64-a1f3-306cd512ae81") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:40.487604 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:40.487527    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:40.487767 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:40.487528    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:04:40.487767 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:40.487685    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw7gj" podUID="6bbf80a3-a5e2-49a5-ad56-a21747cc4a16"
Jul 28 22:04:40.487767 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:40.487746    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-np5j9" podUID="8497aa3c-a69a-4e64-a1f3-306cd512ae81"
Jul 28 22:04:41.488812 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:41.488302    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:41.488812 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:41.488434    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvqcn" podUID="f738f750-e0f4-49e7-b7e6-0c151e664ce9"
Jul 28 22:04:42.488466 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:42.488427    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:04:42.488602 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:42.488545    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-np5j9" podUID="8497aa3c-a69a-4e64-a1f3-306cd512ae81"
Jul 28 22:04:42.488658 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:42.488603    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:42.488720 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:42.488702    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw7gj" podUID="6bbf80a3-a5e2-49a5-ad56-a21747cc4a16"
Jul 28 22:04:43.487937 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:43.487907    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:43.488328 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:43.488056    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvqcn" podUID="f738f750-e0f4-49e7-b7e6-0c151e664ce9"
Jul 28 22:04:44.488291 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:44.488251    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:04:44.488291 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:44.488251    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:44.488821 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:44.488528    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw7gj" podUID="6bbf80a3-a5e2-49a5-ad56-a21747cc4a16"
Jul 28 22:04:44.488821 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:44.488612    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-np5j9" podUID="8497aa3c-a69a-4e64-a1f3-306cd512ae81"
Jul 28 22:04:45.487888 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:45.487849    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:45.488049 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:45.487980    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvqcn" podUID="f738f750-e0f4-49e7-b7e6-0c151e664ce9"
Jul 28 22:04:46.487852 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:46.487828    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:04:46.488150 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:46.487921    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-np5j9" podUID="8497aa3c-a69a-4e64-a1f3-306cd512ae81"
Jul 28 22:04:46.488150 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:46.487946    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:46.488150 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:46.487997    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw7gj" podUID="6bbf80a3-a5e2-49a5-ad56-a21747cc4a16"
Jul 28 22:04:47.487602 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:47.487569    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:47.487793 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:47.487731    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvqcn" podUID="f738f750-e0f4-49e7-b7e6-0c151e664ce9"
Jul 28 22:04:48.154824 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:48.154790    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret\") pod \"global-pull-secret-syncer-qw7gj\" (UID: \"6bbf80a3-a5e2-49a5-ad56-a21747cc4a16\") " pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:48.155322 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:48.154862    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs\") pod \"network-metrics-daemon-dvqcn\" (UID: \"f738f750-e0f4-49e7-b7e6-0c151e664ce9\") " pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:48.155322 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:48.154961    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:48.155322 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:48.154986    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:48.155322 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:48.155034    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret podName:6bbf80a3-a5e2-49a5-ad56-a21747cc4a16 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:04.155014486 +0000 UTC m=+33.139157086 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret") pod "global-pull-secret-syncer-qw7gj" (UID: "6bbf80a3-a5e2-49a5-ad56-a21747cc4a16") : object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:48.155322 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:48.155051    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs podName:f738f750-e0f4-49e7-b7e6-0c151e664ce9 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:04.155044712 +0000 UTC m=+33.139187309 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs") pod "network-metrics-daemon-dvqcn" (UID: "f738f750-e0f4-49e7-b7e6-0c151e664ce9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:48.255168 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:48.255137    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dh6q\" (UniqueName: \"kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q\") pod \"network-check-target-np5j9\" (UID: \"8497aa3c-a69a-4e64-a1f3-306cd512ae81\") " pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:04:48.255347 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:48.255337    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 22:04:48.255394 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:48.255354    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 22:04:48.255394 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:48.255363    2574 projected.go:196] Error preparing data for projected volume kube-api-access-6dh6q for pod openshift-network-diagnostics/network-check-target-np5j9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:48.255458 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:48.255410    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q podName:8497aa3c-a69a-4e64-a1f3-306cd512ae81 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:04.255397336 +0000 UTC m=+33.239539934 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6dh6q" (UniqueName: "kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q") pod "network-check-target-np5j9" (UID: "8497aa3c-a69a-4e64-a1f3-306cd512ae81") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:48.487912 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:48.487877    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:04:48.488117 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:48.487893    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:48.488117 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:48.487998    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-np5j9" podUID="8497aa3c-a69a-4e64-a1f3-306cd512ae81"
Jul 28 22:04:48.488117 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:48.488099    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw7gj" podUID="6bbf80a3-a5e2-49a5-ad56-a21747cc4a16"
Jul 28 22:04:49.488517 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:49.488481    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:49.488972 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:49.488630    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvqcn" podUID="f738f750-e0f4-49e7-b7e6-0c151e664ce9"
Jul 28 22:04:50.488296 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:50.488261    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:04:50.488440 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:50.488261    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:50.488440 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:50.488387    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-np5j9" podUID="8497aa3c-a69a-4e64-a1f3-306cd512ae81"
Jul 28 22:04:50.488502 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:50.488443    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw7gj" podUID="6bbf80a3-a5e2-49a5-ad56-a21747cc4a16"
Jul 28 22:04:51.488656 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:51.488436    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:51.489158 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:51.488745    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvqcn" podUID="f738f750-e0f4-49e7-b7e6-0c151e664ce9"
Jul 28 22:04:51.608461 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:51.608423    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jrh49" event={"ID":"435f03fc-de40-43d5-a9c9-7f22fb70825a","Type":"ContainerStarted","Data":"04762468cc53c5e546e69b224b3d092a7f2c765a8b13e9c7c4b1ff69595b2ed9"}
Jul 28 22:04:51.614816 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:51.614696    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-97vh9" event={"ID":"b505b3f1-0db6-4d26-94c8-fdc9637e3415","Type":"ContainerStarted","Data":"b03484126078fe958cf97ba11980c71804dc27fc4962fb23ff4d351e8a4b1317"}
Jul 28 22:04:51.624666 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:51.621038    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-69.ec2.internal" event={"ID":"7cc3a9f08dccc44aae468666893de9e2","Type":"ContainerStarted","Data":"1886c71ed19639fc04e81eb63afc30258c9e07bab2a5b581f83e6ce11252e990"}
Jul 28 22:04:51.632990 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:51.632399    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-97vh9" podStartSLOduration=2.145162391 podStartE2EDuration="20.632376603s" totalImagesPullingTime="18.487214212s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:04:51.631963832 +0000 UTC m=+20.616106452" watchObservedRunningTime="2026-07-28 22:04:51.632376603 +0000 UTC m=+20.616519222"
Jul 28 22:04:51.650039 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:51.649721    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-69.ec2.internal" podStartSLOduration=19.649701128 podStartE2EDuration="19.649701128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:04:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:04:51.648904544 +0000 UTC m=+20.633047166" watchObservedRunningTime="2026-07-28 22:04:51.649701128 +0000 UTC m=+20.633843747"
Jul 28 22:04:52.487888 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:52.487857    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:04:52.488025 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:52.487863    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:52.488025 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:52.487963    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-np5j9" podUID="8497aa3c-a69a-4e64-a1f3-306cd512ae81"
Jul 28 22:04:52.488107 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:52.488055    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw7gj" podUID="6bbf80a3-a5e2-49a5-ad56-a21747cc4a16"
Jul 28 22:04:52.624997 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:52.624769    2574 generic.go:356] "Generic (PLEG): container finished" podID="1d63809c-6d3f-4a78-9c1c-24b2cf6223f4" containerID="07aa0844ba812d31332d5b1b4ee14bf91937400e30dac5aef3ed02a95d5c57c3" exitCode=0
Jul 28 22:04:52.625687 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:52.624840    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mj477" event={"ID":"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4","Type":"ContainerDied","Data":"07aa0844ba812d31332d5b1b4ee14bf91937400e30dac5aef3ed02a95d5c57c3"}
Jul 28 22:04:52.627718 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:52.627695    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jrh49" event={"ID":"435f03fc-de40-43d5-a9c9-7f22fb70825a","Type":"ContainerStarted","Data":"a85540c7f293e35840c75198b0c85351aaa6d4e579546f868b659511445dc2e2"}
Jul 28 22:04:52.627809 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:52.627759    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jrh49" event={"ID":"435f03fc-de40-43d5-a9c9-7f22fb70825a","Type":"ContainerStarted","Data":"4652cce8373488ac981b3437bc21a786e487dab0d991bf25e5fbfbf2b815cb19"}
Jul 28 22:04:52.627809 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:52.627790    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jrh49" event={"ID":"435f03fc-de40-43d5-a9c9-7f22fb70825a","Type":"ContainerStarted","Data":"841d58610d6bf084adbafaee47cffd7f296d0a4e385d13d38a877f81cd1f1664"}
Jul 28 22:04:52.627896 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:52.627828    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jrh49" event={"ID":"435f03fc-de40-43d5-a9c9-7f22fb70825a","Type":"ContainerStarted","Data":"6928cceb83f6721fc9d1560d209fe5110808b8bfec5d8354f993cf8d277d42e9"}
Jul 28 22:04:52.629154 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:52.629131    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bzhk8" event={"ID":"c6e1d829-065c-4516-a1cf-456e7d3ff8c1","Type":"ContainerStarted","Data":"195d6039d9dc258169085d0fb05f7b0e8b6546d1bc88dcd6df68f83f43d5c99e"}
Jul 28 22:04:52.630675 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:52.630655    2574 generic.go:356] "Generic (PLEG): container finished" podID="54f8ab6a0dce2d047f0664fad06551cb" containerID="66cf953877978d12d0edefe71a07bf36b4ba165ab324f78918f3be4791986680" exitCode=0
Jul 28 22:04:52.630769 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:52.630734    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-69.ec2.internal" event={"ID":"54f8ab6a0dce2d047f0664fad06551cb","Type":"ContainerDied","Data":"66cf953877978d12d0edefe71a07bf36b4ba165ab324f78918f3be4791986680"}
Jul 28 22:04:52.632544 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:52.632261    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-swmbg" event={"ID":"da509f2e-8338-4ece-9b46-ef348ba52ebc","Type":"ContainerStarted","Data":"078ff5f5df3ac16252520c8558a496a14c6ca7044d2cb48717991aaaa6f8f657"}
Jul 28 22:04:52.633682 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:52.633663    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d" event={"ID":"fa48de0c-c971-487e-8bb8-c5e500ea2a33","Type":"ContainerStarted","Data":"49035a8c77f144943d62391112890671cdeefa2c308460e695b61f4986caed00"}
Jul 28 22:04:52.635004 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:52.634976    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5vhbh" event={"ID":"c7cde400-9ef6-4b09-9cc5-f1d05d424e6c","Type":"ContainerStarted","Data":"ef9a09db2d6e4c056da6a9107bfbdeb468c8ab54deaacba70eaa42c5b4ed86c4"}
Jul 28 22:04:52.636380 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:52.636226    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gnhk8" event={"ID":"0efef71b-3b20-4214-914d-5138f89c14fc","Type":"ContainerStarted","Data":"ea1c2629d7e41090cf71ff101565bdfcda3822b3d069de3204810d1b4a21e8fa"}
Jul 28 22:04:52.677522 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:52.677469    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-swmbg" podStartSLOduration=3.173982483 podStartE2EDuration="21.677452858s" totalImagesPullingTime="18.503470375s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:04:52.663071405 +0000 UTC m=+21.647214025" watchObservedRunningTime="2026-07-28 22:04:52.677452858 +0000 UTC m=+21.661595478"
Jul 28 22:04:52.710341 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:52.710288    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bzhk8" podStartSLOduration=7.8835619 podStartE2EDuration="21.710267727s" totalImagesPullingTime="13.826705827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:04:52.709861222 +0000 UTC m=+21.694003842" watchObservedRunningTime="2026-07-28 22:04:52.710267727 +0000 UTC m=+21.694410347"
Jul 28 22:04:52.710509 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:52.710422    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gnhk8" podStartSLOduration=3.266372957 podStartE2EDuration="21.710417268s" totalImagesPullingTime="18.444044311s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:04:52.690930991 +0000 UTC m=+21.675073612" watchObservedRunningTime="2026-07-28 22:04:52.710417268 +0000 UTC m=+21.694559887"
Jul 28 22:04:52.725702 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:52.725657    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5vhbh" podStartSLOduration=2.345277303 podStartE2EDuration="21.725641805s" totalImagesPullingTime="19.380364502s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:04:52.725559181 +0000 UTC m=+21.709701811" watchObservedRunningTime="2026-07-28 22:04:52.725641805 +0000 UTC m=+21.709784427"
Jul 28 22:04:52.893072 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:52.893046    2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 22:04:53.397612 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:53.397495    2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T22:04:52.893064867Z","UUID":"a381472b-ef33-4f6c-bb7f-45410252dbc7","Handler":null,"Name":"","Endpoint":""}
Jul 28 22:04:53.399395 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:53.399366    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 22:04:53.399582 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:53.399407    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 22:04:53.488548 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:53.488515    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:53.488736 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:53.488659    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvqcn" podUID="f738f750-e0f4-49e7-b7e6-0c151e664ce9"
Jul 28 22:04:53.641141 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:53.641101    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-69.ec2.internal" event={"ID":"54f8ab6a0dce2d047f0664fad06551cb","Type":"ContainerStarted","Data":"f1e6490e05ea3ecd546331f978ab64e0add45627f523dbac9bc331eac7c1cf35"}
Jul 28 22:04:53.643559 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:53.643531    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d" event={"ID":"fa48de0c-c971-487e-8bb8-c5e500ea2a33","Type":"ContainerStarted","Data":"eae92d250ab4ca999e316003658da367757642707bf5e6b248d93adfb5d53ca7"}
Jul 28 22:04:53.646970 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:53.646941    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jrh49" event={"ID":"435f03fc-de40-43d5-a9c9-7f22fb70825a","Type":"ContainerStarted","Data":"665d1795fd954c02b6e41bc6f2edf7f27912f9b58327b99db272fda549701062"}
Jul 28 22:04:53.648494 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:53.648427    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-26gxp" event={"ID":"d7a52716-3aa2-425f-bd5b-40184621f385","Type":"ContainerStarted","Data":"306c084993aca2f4895fcb5c0ab5eb021e0ea2c06df1e82e9e34d61e74481f1e"}
Jul 28 22:04:53.659546 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:53.659497    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-69.ec2.internal" podStartSLOduration=21.659483332 podStartE2EDuration="21.659483332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:04:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:04:53.659261709 +0000 UTC m=+22.643404330" watchObservedRunningTime="2026-07-28 22:04:53.659483332 +0000 UTC m=+22.643625952"
Jul 28 22:04:53.694741 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:53.694682    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-26gxp" podStartSLOduration=4.2449636 podStartE2EDuration="22.694653878s" totalImagesPullingTime="18.449690278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:04:53.694308056 +0000 UTC m=+22.678450676" watchObservedRunningTime="2026-07-28 22:04:53.694653878 +0000 UTC m=+22.678796499"
Jul 28 22:04:54.488012 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:54.487982    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:54.488399 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:54.488110    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw7gj" podUID="6bbf80a3-a5e2-49a5-ad56-a21747cc4a16"
Jul 28 22:04:54.488399 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:54.488185    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:04:54.488399 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:54.488326    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-np5j9" podUID="8497aa3c-a69a-4e64-a1f3-306cd512ae81"
Jul 28 22:04:55.487577 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:55.487545    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:55.487985 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:55.487661    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvqcn" podUID="f738f750-e0f4-49e7-b7e6-0c151e664ce9"
Jul 28 22:04:55.656935 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:55.656894    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jrh49" event={"ID":"435f03fc-de40-43d5-a9c9-7f22fb70825a","Type":"ContainerStarted","Data":"bc2d6fb1c116b62c2b843cff2ce1c35578a8fc815b30adf77787b88ae5606bef"}
Jul 28 22:04:55.659183 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:55.659153    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d" event={"ID":"fa48de0c-c971-487e-8bb8-c5e500ea2a33","Type":"ContainerStarted","Data":"e653bc2d4f8581ea5a45f3cfbd79d826afb3abea0a9233022f88481a130d63fa"}
Jul 28 22:04:55.676459 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:55.676399    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9m6d" podStartSLOduration=3.161715504 podStartE2EDuration="24.676378358s" totalImagesPullingTime="21.514662854s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:04:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:04:55.676098822 +0000 UTC m=+24.660241479" watchObservedRunningTime="2026-07-28 22:04:55.676378358 +0000 UTC m=+24.660520977"
Jul 28 22:04:56.188693 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:56.188662    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bzhk8"
Jul 28 22:04:56.189306 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:56.189280    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bzhk8"
Jul 28 22:04:56.488371 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:56.488334    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:56.488371 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:56.488361    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:04:56.488867 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:56.488457    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw7gj" podUID="6bbf80a3-a5e2-49a5-ad56-a21747cc4a16"
Jul 28 22:04:56.488867 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:56.488609    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-np5j9" podUID="8497aa3c-a69a-4e64-a1f3-306cd512ae81"
Jul 28 22:04:56.661886 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:56.661855    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bzhk8"
Jul 28 22:04:56.662360 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:56.662341    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bzhk8"
Jul 28 22:04:57.488451 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:57.488228    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:57.488954 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:57.488544    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvqcn" podUID="f738f750-e0f4-49e7-b7e6-0c151e664ce9"
Jul 28 22:04:57.665371 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:57.665337    2574 generic.go:356] "Generic (PLEG): container finished" podID="1d63809c-6d3f-4a78-9c1c-24b2cf6223f4" containerID="57f09ec2a015e989e42d7b05e812a3c943ae524fbe017a9be492344fb16f1e02" exitCode=0
Jul 28 22:04:57.665517 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:57.665410    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mj477" event={"ID":"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4","Type":"ContainerDied","Data":"57f09ec2a015e989e42d7b05e812a3c943ae524fbe017a9be492344fb16f1e02"}
Jul 28 22:04:57.669193 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:57.669165    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jrh49" event={"ID":"435f03fc-de40-43d5-a9c9-7f22fb70825a","Type":"ContainerStarted","Data":"d13f7b973a91cddae6957e44cbf771cbbe1cc9732032d3a2c54525d30441d1f3"}
Jul 28 22:04:57.747927 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:57.747827    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jrh49" podStartSLOduration=8.190331385 podStartE2EDuration="26.7478108s" totalImagesPullingTime="18.557479415s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:04:57.747334291 +0000 UTC m=+26.731476911" watchObservedRunningTime="2026-07-28 22:04:57.7478108 +0000 UTC m=+26.731953419"
Jul 28 22:04:58.488120 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:58.488081    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:58.488314 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:58.488224    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw7gj" podUID="6bbf80a3-a5e2-49a5-ad56-a21747cc4a16"
Jul 28 22:04:58.488691 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:58.488668    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:04:58.489054 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:58.488770    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-np5j9" podUID="8497aa3c-a69a-4e64-a1f3-306cd512ae81"
Jul 28 22:04:58.672290 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:58.672252    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:58.672453 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:58.672325    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:58.672453 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:58.672362    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:58.686768 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:58.686735    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:58.686955 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:58.686941    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:04:59.488101 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:59.487913    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:59.488262 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:59.488181    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvqcn" podUID="f738f750-e0f4-49e7-b7e6-0c151e664ce9"
Jul 28 22:04:59.554368 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:59.554338    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qw7gj"]
Jul 28 22:04:59.554999 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:59.554452    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:04:59.554999 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:59.554551    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw7gj" podUID="6bbf80a3-a5e2-49a5-ad56-a21747cc4a16"
Jul 28 22:04:59.559040 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:59.558940    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dvqcn"]
Jul 28 22:04:59.559636 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:59.559611    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-np5j9"]
Jul 28 22:04:59.559731 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:59.559709    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:04:59.559823 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:59.559799    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-np5j9" podUID="8497aa3c-a69a-4e64-a1f3-306cd512ae81"
Jul 28 22:04:59.676439 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:59.676404    2574 generic.go:356] "Generic (PLEG): container finished" podID="1d63809c-6d3f-4a78-9c1c-24b2cf6223f4" containerID="a30b37dd6ebe81936734602f013852dacb4f16ec9681dcdd8483b2af060d5154" exitCode=0
Jul 28 22:04:59.676583 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:59.676491    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:04:59.676583 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:04:59.676483    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mj477" event={"ID":"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4","Type":"ContainerDied","Data":"a30b37dd6ebe81936734602f013852dacb4f16ec9681dcdd8483b2af060d5154"}
Jul 28 22:04:59.676682 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:04:59.676582    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvqcn" podUID="f738f750-e0f4-49e7-b7e6-0c151e664ce9"
Jul 28 22:05:01.488185 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:01.488148    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:05:01.488185 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:01.488165    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:05:01.488185 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:01.488148    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:05:01.488662 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:01.488273    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvqcn" podUID="f738f750-e0f4-49e7-b7e6-0c151e664ce9"
Jul 28 22:05:01.488662 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:01.488389    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw7gj" podUID="6bbf80a3-a5e2-49a5-ad56-a21747cc4a16"
Jul 28 22:05:01.488662 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:01.488445    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-np5j9" podUID="8497aa3c-a69a-4e64-a1f3-306cd512ae81"
Jul 28 22:05:01.686251 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:01.686154    2574 generic.go:356] "Generic (PLEG): container finished" podID="1d63809c-6d3f-4a78-9c1c-24b2cf6223f4" containerID="29a78f162ad992f45a01fb3f587e933e85b344ebe34c7ddb82ad1b0532bc4540" exitCode=0
Jul 28 22:05:01.686251 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:01.686204    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mj477" event={"ID":"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4","Type":"ContainerDied","Data":"29a78f162ad992f45a01fb3f587e933e85b344ebe34c7ddb82ad1b0532bc4540"}
Jul 28 22:05:03.250622 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:03.250595    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-swmbg_da509f2e-8338-4ece-9b46-ef348ba52ebc/dns-node-resolver/0.log"
Jul 28 22:05:03.487835 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:03.487804    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:05:03.488023 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:03.487804    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:05:03.488023 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:03.487939    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw7gj" podUID="6bbf80a3-a5e2-49a5-ad56-a21747cc4a16"
Jul 28 22:05:03.488023 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:03.487804    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:05:03.488182 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:03.488043    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvqcn" podUID="f738f750-e0f4-49e7-b7e6-0c151e664ce9"
Jul 28 22:05:03.488182 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:03.488090    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-np5j9" podUID="8497aa3c-a69a-4e64-a1f3-306cd512ae81"
Jul 28 22:05:04.021814 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.021779    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gnhk8_0efef71b-3b20-4214-914d-5138f89c14fc/node-ca/0.log"
Jul 28 22:05:04.177209 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.177157    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret\") pod \"global-pull-secret-syncer-qw7gj\" (UID: \"6bbf80a3-a5e2-49a5-ad56-a21747cc4a16\") " pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:05:04.177498 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.177262    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs\") pod \"network-metrics-daemon-dvqcn\" (UID: \"f738f750-e0f4-49e7-b7e6-0c151e664ce9\") " pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:05:04.177498 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:04.177336    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 22:05:04.177498 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:04.177351    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:05:04.177498 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:04.177416    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret podName:6bbf80a3-a5e2-49a5-ad56-a21747cc4a16 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:36.177396076 +0000 UTC m=+65.161538674 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret") pod "global-pull-secret-syncer-qw7gj" (UID: "6bbf80a3-a5e2-49a5-ad56-a21747cc4a16") : object "kube-system"/"original-pull-secret" not registered
Jul 28 22:05:04.177498 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:04.177434    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs podName:f738f750-e0f4-49e7-b7e6-0c151e664ce9 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:36.177426443 +0000 UTC m=+65.161569054 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs") pod "network-metrics-daemon-dvqcn" (UID: "f738f750-e0f4-49e7-b7e6-0c151e664ce9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:05:04.278356 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.278273    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dh6q\" (UniqueName: \"kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q\") pod \"network-check-target-np5j9\" (UID: \"8497aa3c-a69a-4e64-a1f3-306cd512ae81\") " pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:05:04.278771 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:04.278436    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 22:05:04.278771 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:04.278454    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 22:05:04.278771 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:04.278464    2574 projected.go:196] Error preparing data for projected volume kube-api-access-6dh6q for pod openshift-network-diagnostics/network-check-target-np5j9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:05:04.278771 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:04.278514    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q podName:8497aa3c-a69a-4e64-a1f3-306cd512ae81 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:36.278500504 +0000 UTC m=+65.262643105 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6dh6q" (UniqueName: "kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q") pod "network-check-target-np5j9" (UID: "8497aa3c-a69a-4e64-a1f3-306cd512ae81") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:05:04.835250 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.835201    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-69.ec2.internal" event="NodeReady"
Jul 28 22:05:04.835432 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.835376    2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 22:05:04.868047 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.868005    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-66bff7f44f-mznc6"]
Jul 28 22:05:04.870609 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.870582    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:04.872844 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.872812    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 22:05:04.872987 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.872870    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 22:05:04.873213 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.873183    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 22:05:04.874129 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.873720    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2lgj6\""
Jul 28 22:05:04.879854 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.879826    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hqgs4"]
Jul 28 22:05:04.881634 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.881608    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 22:05:04.882073 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.882049    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hqgs4"
Jul 28 22:05:04.884566 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.884544    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 22:05:04.886572 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.886262    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66bff7f44f-mznc6"]
Jul 28 22:05:04.886572 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.886367    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8xbs4\""
Jul 28 22:05:04.886572 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.886430    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 22:05:04.887221 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.887196    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-n8ks7"]
Jul 28 22:05:04.891704 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.891684    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hqgs4"]
Jul 28 22:05:04.891814 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.891802    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n8ks7"
Jul 28 22:05:04.894048 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.893759    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zn58b\""
Jul 28 22:05:04.894048 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.893976    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 22:05:04.894048 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.893982    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 22:05:04.894377 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.894360    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 22:05:04.906307 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.906280    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lhf6r"]
Jul 28 22:05:04.908569 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.908542    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:04.910114 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.910087    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n8ks7"]
Jul 28 22:05:04.911469 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.911447    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 22:05:04.912099 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.912078    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 22:05:04.912207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.912085    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8gj46\""
Jul 28 22:05:04.912431 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.912410    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 22:05:04.912586 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.912521    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 22:05:04.920527 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.920503    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lhf6r"]
Jul 28 22:05:04.983834 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.983798    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b5d9fd68-fbef-4b56-ac86-4b26358778e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:04.983834 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.983840    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4c33216f-bedb-44c6-b91c-2b84dabc9f77-cert\") pod \"ingress-canary-n8ks7\" (UID: \"4c33216f-bedb-44c6-b91c-2b84dabc9f77\") " pod="openshift-ingress-canary/ingress-canary-n8ks7"
Jul 28 22:05:04.984083 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.983884    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-metrics-tls\") pod \"dns-default-hqgs4\" (UID: \"e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4\") " pod="openshift-dns/dns-default-hqgs4"
Jul 28 22:05:04.984083 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.983932    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6gjk\" (UniqueName: \"kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-kube-api-access-m6gjk\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:04.984083 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.983951    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b5d9fd68-fbef-4b56-ac86-4b26358778e7-data-volume\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:04.984083 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.983966    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrvkz\" (UniqueName: \"kubernetes.io/projected/b5d9fd68-fbef-4b56-ac86-4b26358778e7-kube-api-access-qrvkz\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:04.984083 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.983987    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-tls\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:04.984367 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.984084    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bd70751c-6f8d-4c3e-9602-cfa827b322f3-image-registry-private-configuration\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:04.984367 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.984111    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bd70751c-6f8d-4c3e-9602-cfa827b322f3-ca-trust-extracted\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:04.984367 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.984131    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bwvp\" (UniqueName: \"kubernetes.io/projected/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-kube-api-access-7bwvp\") pod \"dns-default-hqgs4\" (UID: \"e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4\") " pod="openshift-dns/dns-default-hqgs4"
Jul 28 22:05:04.984367 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.984152    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6kp6\" (UniqueName: \"kubernetes.io/projected/4c33216f-bedb-44c6-b91c-2b84dabc9f77-kube-api-access-t6kp6\") pod \"ingress-canary-n8ks7\" (UID: \"4c33216f-bedb-44c6-b91c-2b84dabc9f77\") " pod="openshift-ingress-canary/ingress-canary-n8ks7"
Jul 28 22:05:04.984367 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.984187    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-config-volume\") pod \"dns-default-hqgs4\" (UID: \"e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4\") " pod="openshift-dns/dns-default-hqgs4"
Jul 28 22:05:04.984367 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.984206    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b5d9fd68-fbef-4b56-ac86-4b26358778e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:04.984367 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.984254    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-tmp-dir\") pod \"dns-default-hqgs4\" (UID: \"e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4\") " pod="openshift-dns/dns-default-hqgs4"
Jul 28 22:05:04.984367 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.984288    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bd70751c-6f8d-4c3e-9602-cfa827b322f3-trusted-ca\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:04.984367 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.984311    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bd70751c-6f8d-4c3e-9602-cfa827b322f3-installation-pull-secrets\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:04.984367 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.984341    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-certificates\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:04.984367 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.984364    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-bound-sa-token\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:04.984894 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:04.984382    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b5d9fd68-fbef-4b56-ac86-4b26358778e7-crio-socket\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:05.085068 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.085032    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b5d9fd68-fbef-4b56-ac86-4b26358778e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:05.085068 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.085069    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4c33216f-bedb-44c6-b91c-2b84dabc9f77-cert\") pod \"ingress-canary-n8ks7\" (UID: \"4c33216f-bedb-44c6-b91c-2b84dabc9f77\") " pod="openshift-ingress-canary/ingress-canary-n8ks7"
Jul 28 22:05:05.085390 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.085162    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-metrics-tls\") pod \"dns-default-hqgs4\" (UID: \"e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4\") " pod="openshift-dns/dns-default-hqgs4"
Jul 28 22:05:05.085390 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.085192    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6gjk\" (UniqueName: \"kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-kube-api-access-m6gjk\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:05.085390 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:05.085202    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 22:05:05.085390 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.085220    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b5d9fd68-fbef-4b56-ac86-4b26358778e7-data-volume\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:05.085390 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.085255    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrvkz\" (UniqueName: \"kubernetes.io/projected/b5d9fd68-fbef-4b56-ac86-4b26358778e7-kube-api-access-qrvkz\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:05.085390 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:05.085297    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5d9fd68-fbef-4b56-ac86-4b26358778e7-insights-runtime-extractor-tls podName:b5d9fd68-fbef-4b56-ac86-4b26358778e7 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:05.585266676 +0000 UTC m=+34.569409297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b5d9fd68-fbef-4b56-ac86-4b26358778e7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lhf6r" (UID: "b5d9fd68-fbef-4b56-ac86-4b26358778e7") : secret "insights-runtime-extractor-tls" not found
Jul 28 22:05:05.085390 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:05.085328    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 22:05:05.085390 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.085336    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-tls\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:05.085390 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:05.085391    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-metrics-tls podName:e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:05.585377622 +0000 UTC m=+34.569520220 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-metrics-tls") pod "dns-default-hqgs4" (UID: "e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4") : secret "dns-default-metrics-tls" not found
Jul 28 22:05:05.085851 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.085409    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bd70751c-6f8d-4c3e-9602-cfa827b322f3-image-registry-private-configuration\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:05.085851 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.085431    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bd70751c-6f8d-4c3e-9602-cfa827b322f3-ca-trust-extracted\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:05.085851 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.085456    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bwvp\" (UniqueName: \"kubernetes.io/projected/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-kube-api-access-7bwvp\") pod \"dns-default-hqgs4\" (UID: \"e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4\") " pod="openshift-dns/dns-default-hqgs4"
Jul 28 22:05:05.085851 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.085479    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6kp6\" (UniqueName: \"kubernetes.io/projected/4c33216f-bedb-44c6-b91c-2b84dabc9f77-kube-api-access-t6kp6\") pod \"ingress-canary-n8ks7\" (UID: \"4c33216f-bedb-44c6-b91c-2b84dabc9f77\") " pod="openshift-ingress-canary/ingress-canary-n8ks7"
Jul 28 22:05:05.085851 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.085553    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-config-volume\") pod \"dns-default-hqgs4\" (UID: \"e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4\") " pod="openshift-dns/dns-default-hqgs4"
Jul 28 22:05:05.085851 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.085570    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b5d9fd68-fbef-4b56-ac86-4b26358778e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:05.085851 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.085590    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-tmp-dir\") pod \"dns-default-hqgs4\" (UID: \"e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4\") " pod="openshift-dns/dns-default-hqgs4"
Jul 28 22:05:05.085851 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.085600    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b5d9fd68-fbef-4b56-ac86-4b26358778e7-data-volume\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:05.085851 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.085616    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bd70751c-6f8d-4c3e-9602-cfa827b322f3-trusted-ca\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:05.085851 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.085631    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bd70751c-6f8d-4c3e-9602-cfa827b322f3-installation-pull-secrets\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:05.085851 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.085649    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-certificates\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:05.085851 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.085675    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-bound-sa-token\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:05.085851 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.085700    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b5d9fd68-fbef-4b56-ac86-4b26358778e7-crio-socket\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:05.085851 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.085820    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bd70751c-6f8d-4c3e-9602-cfa827b322f3-ca-trust-extracted\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:05.085851 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:05.085432    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 22:05:05.085851 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.085850    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b5d9fd68-fbef-4b56-ac86-4b26358778e7-crio-socket\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:05.086624 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.086099    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-tmp-dir\") pod \"dns-default-hqgs4\" (UID: \"e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4\") " pod="openshift-dns/dns-default-hqgs4"
Jul 28 22:05:05.086624 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:05.085853    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66bff7f44f-mznc6: secret "image-registry-tls" not found
Jul 28 22:05:05.086624 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.086434    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-certificates\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:05.086624 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:05.085835    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 22:05:05.086624 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:05.086466    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-tls podName:bd70751c-6f8d-4c3e-9602-cfa827b322f3 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:05.586449381 +0000 UTC m=+34.570591999 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-tls") pod "image-registry-66bff7f44f-mznc6" (UID: "bd70751c-6f8d-4c3e-9602-cfa827b322f3") : secret "image-registry-tls" not found
Jul 28 22:05:05.086624 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:05.086536    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b5d9fd68-fbef-4b56-ac86-4b26358778e7-kube-rbac-proxy-cm podName:b5d9fd68-fbef-4b56-ac86-4b26358778e7 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:05.586520673 +0000 UTC m=+34.570663271 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b5d9fd68-fbef-4b56-ac86-4b26358778e7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lhf6r" (UID: "b5d9fd68-fbef-4b56-ac86-4b26358778e7") : configmap "kube-rbac-proxy" not found
Jul 28 22:05:05.086624 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.086591    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-config-volume\") pod \"dns-default-hqgs4\" (UID: \"e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4\") " pod="openshift-dns/dns-default-hqgs4"
Jul 28 22:05:05.087361 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.087331    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bd70751c-6f8d-4c3e-9602-cfa827b322f3-trusted-ca\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:05.090882 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.090753    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bd70751c-6f8d-4c3e-9602-cfa827b322f3-installation-pull-secrets\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:05.091015 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.090766    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4c33216f-bedb-44c6-b91c-2b84dabc9f77-cert\") pod \"ingress-canary-n8ks7\" (UID: \"4c33216f-bedb-44c6-b91c-2b84dabc9f77\") " pod="openshift-ingress-canary/ingress-canary-n8ks7"
Jul 28 22:05:05.091162 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.091142    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bd70751c-6f8d-4c3e-9602-cfa827b322f3-image-registry-private-configuration\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:05.099688 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.099652    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrvkz\" (UniqueName: \"kubernetes.io/projected/b5d9fd68-fbef-4b56-ac86-4b26358778e7-kube-api-access-qrvkz\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:05.100088 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.100066    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bwvp\" (UniqueName: \"kubernetes.io/projected/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-kube-api-access-7bwvp\") pod \"dns-default-hqgs4\" (UID: \"e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4\") " pod="openshift-dns/dns-default-hqgs4"
Jul 28 22:05:05.100088 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.100074    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-bound-sa-token\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:05.100269 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.100155    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6gjk\" (UniqueName: \"kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-kube-api-access-m6gjk\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:05.100911 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.100889    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6kp6\" (UniqueName: \"kubernetes.io/projected/4c33216f-bedb-44c6-b91c-2b84dabc9f77-kube-api-access-t6kp6\") pod \"ingress-canary-n8ks7\" (UID: \"4c33216f-bedb-44c6-b91c-2b84dabc9f77\") " pod="openshift-ingress-canary/ingress-canary-n8ks7"
Jul 28 22:05:05.203742 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.203703    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n8ks7"
Jul 28 22:05:05.488140 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.488105    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:05:05.488594 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.488106    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:05:05.488594 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.488106    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:05:05.493730 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.492145    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sm6xq\""
Jul 28 22:05:05.493730 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.492172    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 22:05:05.493730 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.492506    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bnqzv\""
Jul 28 22:05:05.493730 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.492697    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 22:05:05.493730 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.492820    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 22:05:05.493730 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.492959    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 22:05:05.589381 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.589339    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-metrics-tls\") pod \"dns-default-hqgs4\" (UID: \"e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4\") " pod="openshift-dns/dns-default-hqgs4"
Jul 28 22:05:05.589575 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.589403    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-tls\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:05.589575 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.589472    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b5d9fd68-fbef-4b56-ac86-4b26358778e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:05.589575 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.589522    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b5d9fd68-fbef-4b56-ac86-4b26358778e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:05.589575 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:05.589522    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 22:05:05.589575 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:05.589561    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 22:05:05.589818 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:05.589581    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66bff7f44f-mznc6: secret "image-registry-tls" not found
Jul 28 22:05:05.589818 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:05.589615    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 22:05:05.589818 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:05.589633    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 22:05:05.589818 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:05.589639    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-metrics-tls podName:e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:06.589618705 +0000 UTC m=+35.573761304 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-metrics-tls") pod "dns-default-hqgs4" (UID: "e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4") : secret "dns-default-metrics-tls" not found
Jul 28 22:05:05.589818 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:05.589701    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-tls podName:bd70751c-6f8d-4c3e-9602-cfa827b322f3 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:06.589682851 +0000 UTC m=+35.573825456 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-tls") pod "image-registry-66bff7f44f-mznc6" (UID: "bd70751c-6f8d-4c3e-9602-cfa827b322f3") : secret "image-registry-tls" not found
Jul 28 22:05:05.589818 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:05.589719    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b5d9fd68-fbef-4b56-ac86-4b26358778e7-kube-rbac-proxy-cm podName:b5d9fd68-fbef-4b56-ac86-4b26358778e7 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:06.589707994 +0000 UTC m=+35.573850591 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b5d9fd68-fbef-4b56-ac86-4b26358778e7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lhf6r" (UID: "b5d9fd68-fbef-4b56-ac86-4b26358778e7") : configmap "kube-rbac-proxy" not found
Jul 28 22:05:05.589818 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:05.589737    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5d9fd68-fbef-4b56-ac86-4b26358778e7-insights-runtime-extractor-tls podName:b5d9fd68-fbef-4b56-ac86-4b26358778e7 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:06.589726491 +0000 UTC m=+35.573869090 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b5d9fd68-fbef-4b56-ac86-4b26358778e7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lhf6r" (UID: "b5d9fd68-fbef-4b56-ac86-4b26358778e7") : secret "insights-runtime-extractor-tls" not found
Jul 28 22:05:05.796665 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:05.796641    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n8ks7"]
Jul 28 22:05:05.801325 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:05:05.801295    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c33216f_bedb_44c6_b91c_2b84dabc9f77.slice/crio-68ac8e4e3acfd708874d70d6fd8d0c6e4da25566b92515fb125e78b49e0f6e26 WatchSource:0}: Error finding container 68ac8e4e3acfd708874d70d6fd8d0c6e4da25566b92515fb125e78b49e0f6e26: Status 404 returned error can't find the container with id 68ac8e4e3acfd708874d70d6fd8d0c6e4da25566b92515fb125e78b49e0f6e26
Jul 28 22:05:06.596886 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:06.596851    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b5d9fd68-fbef-4b56-ac86-4b26358778e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:06.597707 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:06.596897    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b5d9fd68-fbef-4b56-ac86-4b26358778e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:06.597707 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:06.596980    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 22:05:06.597707 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:06.597043    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b5d9fd68-fbef-4b56-ac86-4b26358778e7-kube-rbac-proxy-cm podName:b5d9fd68-fbef-4b56-ac86-4b26358778e7 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:08.597028587 +0000 UTC m=+37.581171185 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b5d9fd68-fbef-4b56-ac86-4b26358778e7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lhf6r" (UID: "b5d9fd68-fbef-4b56-ac86-4b26358778e7") : configmap "kube-rbac-proxy" not found
Jul 28 22:05:06.597707 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:06.597037    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-metrics-tls\") pod \"dns-default-hqgs4\" (UID: \"e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4\") " pod="openshift-dns/dns-default-hqgs4"
Jul 28 22:05:06.597707 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:06.597085    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-tls\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:06.597707 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:06.597106    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 22:05:06.597707 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:06.597131    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 22:05:06.597707 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:06.597149    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-metrics-tls podName:e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:08.597138307 +0000 UTC m=+37.581280904 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-metrics-tls") pod "dns-default-hqgs4" (UID: "e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4") : secret "dns-default-metrics-tls" not found
Jul 28 22:05:06.597707 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:06.597166    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5d9fd68-fbef-4b56-ac86-4b26358778e7-insights-runtime-extractor-tls podName:b5d9fd68-fbef-4b56-ac86-4b26358778e7 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:08.597156454 +0000 UTC m=+37.581299052 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b5d9fd68-fbef-4b56-ac86-4b26358778e7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lhf6r" (UID: "b5d9fd68-fbef-4b56-ac86-4b26358778e7") : secret "insights-runtime-extractor-tls" not found
Jul 28 22:05:06.597707 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:06.597176    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 22:05:06.597707 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:06.597186    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66bff7f44f-mznc6: secret "image-registry-tls" not found
Jul 28 22:05:06.597707 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:06.597224    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-tls podName:bd70751c-6f8d-4c3e-9602-cfa827b322f3 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:08.597216662 +0000 UTC m=+37.581359259 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-tls") pod "image-registry-66bff7f44f-mznc6" (UID: "bd70751c-6f8d-4c3e-9602-cfa827b322f3") : secret "image-registry-tls" not found
Jul 28 22:05:06.698508 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:06.698474    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n8ks7" event={"ID":"4c33216f-bedb-44c6-b91c-2b84dabc9f77","Type":"ContainerStarted","Data":"68ac8e4e3acfd708874d70d6fd8d0c6e4da25566b92515fb125e78b49e0f6e26"}
Jul 28 22:05:06.701006 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:06.700980    2574 generic.go:356] "Generic (PLEG): container finished" podID="1d63809c-6d3f-4a78-9c1c-24b2cf6223f4" containerID="92b06a2276edd7e95ac4814049337f294c0ca84f14a417e359989452123c3542" exitCode=0
Jul 28 22:05:06.701135 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:06.701029    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mj477" event={"ID":"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4","Type":"ContainerDied","Data":"92b06a2276edd7e95ac4814049337f294c0ca84f14a417e359989452123c3542"}
Jul 28 22:05:07.707896 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:07.707655    2574 generic.go:356] "Generic (PLEG): container finished" podID="1d63809c-6d3f-4a78-9c1c-24b2cf6223f4" containerID="06bdd061116b57a905e5bfd55bcdbc59c62252004b85b03653051a157f4cde0b" exitCode=0
Jul 28 22:05:07.708321 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:07.707734    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mj477" event={"ID":"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4","Type":"ContainerDied","Data":"06bdd061116b57a905e5bfd55bcdbc59c62252004b85b03653051a157f4cde0b"}
Jul 28 22:05:08.612741 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:08.612701    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-tls\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:08.612910 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:08.612878    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 22:05:08.612910 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:08.612903    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66bff7f44f-mznc6: secret "image-registry-tls" not found
Jul 28 22:05:08.613003 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:08.612968    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-tls podName:bd70751c-6f8d-4c3e-9602-cfa827b322f3 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:12.612947282 +0000 UTC m=+41.597089905 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-tls") pod "image-registry-66bff7f44f-mznc6" (UID: "bd70751c-6f8d-4c3e-9602-cfa827b322f3") : secret "image-registry-tls" not found
Jul 28 22:05:08.613051 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:08.612883    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b5d9fd68-fbef-4b56-ac86-4b26358778e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:08.613097 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:08.613078    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b5d9fd68-fbef-4b56-ac86-4b26358778e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:08.613150 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:08.613135    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-metrics-tls\") pod \"dns-default-hqgs4\" (UID: \"e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4\") " pod="openshift-dns/dns-default-hqgs4"
Jul 28 22:05:08.613275 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:08.613260    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 22:05:08.613348 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:08.613316    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-metrics-tls podName:e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:12.613305043 +0000 UTC m=+41.597447641 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-metrics-tls") pod "dns-default-hqgs4" (UID: "e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4") : secret "dns-default-metrics-tls" not found
Jul 28 22:05:08.613348 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:08.613260    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 22:05:08.613348 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:08.613347    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5d9fd68-fbef-4b56-ac86-4b26358778e7-insights-runtime-extractor-tls podName:b5d9fd68-fbef-4b56-ac86-4b26358778e7 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:12.613341093 +0000 UTC m=+41.597483691 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b5d9fd68-fbef-4b56-ac86-4b26358778e7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lhf6r" (UID: "b5d9fd68-fbef-4b56-ac86-4b26358778e7") : secret "insights-runtime-extractor-tls" not found
Jul 28 22:05:08.613506 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:08.613487    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b5d9fd68-fbef-4b56-ac86-4b26358778e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:08.715455 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:08.715414    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mj477" event={"ID":"1d63809c-6d3f-4a78-9c1c-24b2cf6223f4","Type":"ContainerStarted","Data":"04d021d4961473332831f4d212312f5bd5c424ca7713d16e69b53cd3fe7226b5"}
Jul 28 22:05:08.740781 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:08.740714    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mj477" podStartSLOduration=4.809970527 podStartE2EDuration="37.740696934s" totalImagesPullingTime="32.930726407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:04:31 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-28 22:05:08.739799834 +0000 UTC m=+37.723942455" watchObservedRunningTime="2026-07-28 22:05:08.740696934 +0000 UTC m=+37.724839553"
Jul 28 22:05:09.719540 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:09.719502    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n8ks7" event={"ID":"4c33216f-bedb-44c6-b91c-2b84dabc9f77","Type":"ContainerStarted","Data":"36191f0c5a6c0bb7b7525e265ef8f446f5f6fcb4cd28e57133d831ae9051e58b"}
Jul 28 22:05:09.740993 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:09.740938    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-n8ks7" podStartSLOduration=2.639780367 podStartE2EDuration="5.740919833s" totalImagesPullingTime="3.101139466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:05:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:09.739725528 +0000 UTC m=+38.723868170" watchObservedRunningTime="2026-07-28 22:05:09.740919833 +0000 UTC m=+38.725062497"
Jul 28 22:05:12.643559 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:12.643518    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-tls\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:12.643951 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:12.643599    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b5d9fd68-fbef-4b56-ac86-4b26358778e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:12.643951 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:12.643635    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-metrics-tls\") pod \"dns-default-hqgs4\" (UID: \"e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4\") " pod="openshift-dns/dns-default-hqgs4"
Jul 28 22:05:12.643951 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:12.643738    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 22:05:12.643951 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:12.643806    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5d9fd68-fbef-4b56-ac86-4b26358778e7-insights-runtime-extractor-tls podName:b5d9fd68-fbef-4b56-ac86-4b26358778e7 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:20.643788049 +0000 UTC m=+49.627930652 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b5d9fd68-fbef-4b56-ac86-4b26358778e7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lhf6r" (UID: "b5d9fd68-fbef-4b56-ac86-4b26358778e7") : secret "insights-runtime-extractor-tls" not found
Jul 28 22:05:12.646812 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:12.646789    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4-metrics-tls\") pod \"dns-default-hqgs4\" (UID: \"e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4\") " pod="openshift-dns/dns-default-hqgs4"
Jul 28 22:05:12.646939 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:12.646883    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-tls\") pod \"image-registry-66bff7f44f-mznc6\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") " pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:12.687773 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:12.687736    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:12.695879 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:12.695856    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hqgs4"
Jul 28 22:05:12.866109 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:12.866077    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66bff7f44f-mznc6"]
Jul 28 22:05:12.868833 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:05:12.868806    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd70751c_6f8d_4c3e_9602_cfa827b322f3.slice/crio-9dcb39eb1079da7daa36e0afd07606d59018978191a087b4f4e074ee719b5a7c WatchSource:0}: Error finding container 9dcb39eb1079da7daa36e0afd07606d59018978191a087b4f4e074ee719b5a7c: Status 404 returned error can't find the container with id 9dcb39eb1079da7daa36e0afd07606d59018978191a087b4f4e074ee719b5a7c
Jul 28 22:05:12.880870 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:12.880842    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hqgs4"]
Jul 28 22:05:12.885082 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:05:12.885060    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1c1dfc7_3f16_4aa3_b2d9_56caa765dcc4.slice/crio-d1102616823b8c44fb5c37ef9f7a20afd16068160f0d95a55c3ad6ce35bd5d40 WatchSource:0}: Error finding container d1102616823b8c44fb5c37ef9f7a20afd16068160f0d95a55c3ad6ce35bd5d40: Status 404 returned error can't find the container with id d1102616823b8c44fb5c37ef9f7a20afd16068160f0d95a55c3ad6ce35bd5d40
Jul 28 22:05:13.731875 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:13.731839    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hqgs4" event={"ID":"e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4","Type":"ContainerStarted","Data":"d1102616823b8c44fb5c37ef9f7a20afd16068160f0d95a55c3ad6ce35bd5d40"}
Jul 28 22:05:13.733561 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:13.733534    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66bff7f44f-mznc6" event={"ID":"bd70751c-6f8d-4c3e-9602-cfa827b322f3","Type":"ContainerStarted","Data":"4f4f77a506aa9d46a89ee4ccba88d4fd77e9542ac94b0c42a421c1774ebb7a30"}
Jul 28 22:05:13.733692 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:13.733575    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66bff7f44f-mznc6" event={"ID":"bd70751c-6f8d-4c3e-9602-cfa827b322f3","Type":"ContainerStarted","Data":"9dcb39eb1079da7daa36e0afd07606d59018978191a087b4f4e074ee719b5a7c"}
Jul 28 22:05:13.733692 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:13.733649    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:13.753676 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:13.753614    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-66bff7f44f-mznc6" podStartSLOduration=36.753593645 podStartE2EDuration="36.753593645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:04:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:13.752959366 +0000 UTC m=+42.737101986" watchObservedRunningTime="2026-07-28 22:05:13.753593645 +0000 UTC m=+42.737736267"
Jul 28 22:05:15.741546 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:15.741499    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hqgs4" event={"ID":"e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4","Type":"ContainerStarted","Data":"33950c9a61c4296657a9b0a71d66280611cfc88d1cac9d3826e80b6c3bb5131b"}
Jul 28 22:05:16.746343 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:16.746307    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hqgs4" event={"ID":"e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4","Type":"ContainerStarted","Data":"228840dade162dc55d8ddd26e3cf1891c8364d01541a50279959b7e2cd565878"}
Jul 28 22:05:16.746735 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:16.746439    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hqgs4"
Jul 28 22:05:16.776856 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:16.776807    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hqgs4" podStartSLOduration=10.116732586 podStartE2EDuration="12.77679315s" totalImagesPullingTime="2.660060564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:05:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:16.776611948 +0000 UTC m=+45.760754568" watchObservedRunningTime="2026-07-28 22:05:16.77679315 +0000 UTC m=+45.760935769"
Jul 28 22:05:20.707570 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:20.707528    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b5d9fd68-fbef-4b56-ac86-4b26358778e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:20.709799 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:20.709774    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b5d9fd68-fbef-4b56-ac86-4b26358778e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lhf6r\" (UID: \"b5d9fd68-fbef-4b56-ac86-4b26358778e7\") " pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:20.819391 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:20.819358    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lhf6r"
Jul 28 22:05:20.973570 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:20.973540    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lhf6r"]
Jul 28 22:05:21.759936 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:21.759891    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lhf6r" event={"ID":"b5d9fd68-fbef-4b56-ac86-4b26358778e7","Type":"ContainerStarted","Data":"e57b45c4d22be619fb66648d2e1bc2ce5f7a3e1f8c68579860779116bf576df5"}
Jul 28 22:05:25.068097 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.067655    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66bff7f44f-mznc6"]
Jul 28 22:05:25.083496 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.083447    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bd9bc875b-pb4kg"]
Jul 28 22:05:25.104530 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.104497    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c9b9dc685-ftgft"]
Jul 28 22:05:25.104751 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.104645    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bd9bc875b-pb4kg"
Jul 28 22:05:25.109511 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.109151    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"]
Jul 28 22:05:25.109668 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.109589    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c9b9dc685-ftgft"
Jul 28 22:05:25.110327 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.109976    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 28 22:05:25.110327 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.109976    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-fqnts\""
Jul 28 22:05:25.110327 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.110119    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 28 22:05:25.110559 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.110541    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 28 22:05:25.114030 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.114006    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bd9bc875b-pb4kg"]
Jul 28 22:05:25.114154 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.114113    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"
Jul 28 22:05:25.119081 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.119060    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 28 22:05:25.122754 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.122735    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 28 22:05:25.123737 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.123707    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 28 22:05:25.124911 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.124864    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 28 22:05:25.125011 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.124919    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 28 22:05:25.125135 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.125112    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 28 22:05:25.129855 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.129830    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c9b9dc685-ftgft"]
Jul 28 22:05:25.137769 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.137735    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"]
Jul 28 22:05:25.212385 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.212279    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74b5c9c64-f7qff"]
Jul 28 22:05:25.216554 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.216530    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.240436 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.240399    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgw9c\" (UniqueName: \"kubernetes.io/projected/5e7aa6b1-aa47-4a47-8398-5d9cd4b9126b-kube-api-access-jgw9c\") pod \"managed-serviceaccount-addon-agent-bd9bc875b-pb4kg\" (UID: \"5e7aa6b1-aa47-4a47-8398-5d9cd4b9126b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bd9bc875b-pb4kg"
Jul 28 22:05:25.240651 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.240450    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e59d69f3-69b5-4d85-8a54-f9def9627f85-additional-service-ca\") pod \"cluster-proxy-proxy-agent-77bd48578d-jjgt9\" (UID: \"e59d69f3-69b5-4d85-8a54-f9def9627f85\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"
Jul 28 22:05:25.240651 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.240550    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dscx\" (UniqueName: \"kubernetes.io/projected/6e311d08-0827-46f7-8382-9d8ddaaa73de-kube-api-access-4dscx\") pod \"klusterlet-addon-workmgr-5c9b9dc685-ftgft\" (UID: \"6e311d08-0827-46f7-8382-9d8ddaaa73de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c9b9dc685-ftgft"
Jul 28 22:05:25.240651 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.240591    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e59d69f3-69b5-4d85-8a54-f9def9627f85-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-77bd48578d-jjgt9\" (UID: \"e59d69f3-69b5-4d85-8a54-f9def9627f85\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"
Jul 28 22:05:25.240941 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.240664    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5e7aa6b1-aa47-4a47-8398-5d9cd4b9126b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-bd9bc875b-pb4kg\" (UID: \"5e7aa6b1-aa47-4a47-8398-5d9cd4b9126b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bd9bc875b-pb4kg"
Jul 28 22:05:25.240941 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.240730    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e59d69f3-69b5-4d85-8a54-f9def9627f85-hub\") pod \"cluster-proxy-proxy-agent-77bd48578d-jjgt9\" (UID: \"e59d69f3-69b5-4d85-8a54-f9def9627f85\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"
Jul 28 22:05:25.240941 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.240858    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6e311d08-0827-46f7-8382-9d8ddaaa73de-klusterlet-config\") pod \"klusterlet-addon-workmgr-5c9b9dc685-ftgft\" (UID: \"6e311d08-0827-46f7-8382-9d8ddaaa73de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c9b9dc685-ftgft"
Jul 28 22:05:25.240941 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.240896    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e59d69f3-69b5-4d85-8a54-f9def9627f85-ca\") pod \"cluster-proxy-proxy-agent-77bd48578d-jjgt9\" (UID: \"e59d69f3-69b5-4d85-8a54-f9def9627f85\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"
Jul 28 22:05:25.240941 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.240931    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e311d08-0827-46f7-8382-9d8ddaaa73de-tmp\") pod \"klusterlet-addon-workmgr-5c9b9dc685-ftgft\" (UID: \"6e311d08-0827-46f7-8382-9d8ddaaa73de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c9b9dc685-ftgft"
Jul 28 22:05:25.241167 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.241009    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e59d69f3-69b5-4d85-8a54-f9def9627f85-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-77bd48578d-jjgt9\" (UID: \"e59d69f3-69b5-4d85-8a54-f9def9627f85\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"
Jul 28 22:05:25.241167 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.241051    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjb9n\" (UniqueName: \"kubernetes.io/projected/e59d69f3-69b5-4d85-8a54-f9def9627f85-kube-api-access-rjb9n\") pod \"cluster-proxy-proxy-agent-77bd48578d-jjgt9\" (UID: \"e59d69f3-69b5-4d85-8a54-f9def9627f85\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"
Jul 28 22:05:25.254621 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.254591    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74b5c9c64-f7qff"]
Jul 28 22:05:25.341529 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.341442    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6e311d08-0827-46f7-8382-9d8ddaaa73de-klusterlet-config\") pod \"klusterlet-addon-workmgr-5c9b9dc685-ftgft\" (UID: \"6e311d08-0827-46f7-8382-9d8ddaaa73de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c9b9dc685-ftgft"
Jul 28 22:05:25.341529 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.341493    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e59d69f3-69b5-4d85-8a54-f9def9627f85-ca\") pod \"cluster-proxy-proxy-agent-77bd48578d-jjgt9\" (UID: \"e59d69f3-69b5-4d85-8a54-f9def9627f85\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"
Jul 28 22:05:25.341529 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.341522    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e311d08-0827-46f7-8382-9d8ddaaa73de-tmp\") pod \"klusterlet-addon-workmgr-5c9b9dc685-ftgft\" (UID: \"6e311d08-0827-46f7-8382-9d8ddaaa73de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c9b9dc685-ftgft"
Jul 28 22:05:25.341808 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.341553    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e59d69f3-69b5-4d85-8a54-f9def9627f85-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-77bd48578d-jjgt9\" (UID: \"e59d69f3-69b5-4d85-8a54-f9def9627f85\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"
Jul 28 22:05:25.341808 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.341596    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjb9n\" (UniqueName: \"kubernetes.io/projected/e59d69f3-69b5-4d85-8a54-f9def9627f85-kube-api-access-rjb9n\") pod \"cluster-proxy-proxy-agent-77bd48578d-jjgt9\" (UID: \"e59d69f3-69b5-4d85-8a54-f9def9627f85\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"
Jul 28 22:05:25.341808 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.341631    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgw9c\" (UniqueName: \"kubernetes.io/projected/5e7aa6b1-aa47-4a47-8398-5d9cd4b9126b-kube-api-access-jgw9c\") pod \"managed-serviceaccount-addon-agent-bd9bc875b-pb4kg\" (UID: \"5e7aa6b1-aa47-4a47-8398-5d9cd4b9126b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bd9bc875b-pb4kg"
Jul 28 22:05:25.341808 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.341661    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e59d69f3-69b5-4d85-8a54-f9def9627f85-additional-service-ca\") pod \"cluster-proxy-proxy-agent-77bd48578d-jjgt9\" (UID: \"e59d69f3-69b5-4d85-8a54-f9def9627f85\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"
Jul 28 22:05:25.341808 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.341698    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjjzn\" (UniqueName: \"kubernetes.io/projected/e525072e-f5b3-4c68-800c-6292353b55bd-kube-api-access-vjjzn\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.341808 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.341744    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dscx\" (UniqueName: \"kubernetes.io/projected/6e311d08-0827-46f7-8382-9d8ddaaa73de-kube-api-access-4dscx\") pod \"klusterlet-addon-workmgr-5c9b9dc685-ftgft\" (UID: \"6e311d08-0827-46f7-8382-9d8ddaaa73de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c9b9dc685-ftgft"
Jul 28 22:05:25.341808 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.341773    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e59d69f3-69b5-4d85-8a54-f9def9627f85-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-77bd48578d-jjgt9\" (UID: \"e59d69f3-69b5-4d85-8a54-f9def9627f85\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"
Jul 28 22:05:25.341808 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.341807    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e525072e-f5b3-4c68-800c-6292353b55bd-registry-tls\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.342189 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.341841    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e525072e-f5b3-4c68-800c-6292353b55bd-ca-trust-extracted\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.342189 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.341867    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e525072e-f5b3-4c68-800c-6292353b55bd-registry-certificates\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.342189 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.341895    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e525072e-f5b3-4c68-800c-6292353b55bd-bound-sa-token\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.342189 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.341952    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5e7aa6b1-aa47-4a47-8398-5d9cd4b9126b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-bd9bc875b-pb4kg\" (UID: \"5e7aa6b1-aa47-4a47-8398-5d9cd4b9126b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bd9bc875b-pb4kg"
Jul 28 22:05:25.342189 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.341961    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e311d08-0827-46f7-8382-9d8ddaaa73de-tmp\") pod \"klusterlet-addon-workmgr-5c9b9dc685-ftgft\" (UID: \"6e311d08-0827-46f7-8382-9d8ddaaa73de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c9b9dc685-ftgft"
Jul 28 22:05:25.342189 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.341997    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e525072e-f5b3-4c68-800c-6292353b55bd-image-registry-private-configuration\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.342189 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.342050    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e59d69f3-69b5-4d85-8a54-f9def9627f85-hub\") pod \"cluster-proxy-proxy-agent-77bd48578d-jjgt9\" (UID: \"e59d69f3-69b5-4d85-8a54-f9def9627f85\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"
Jul 28 22:05:25.342189 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.342084    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e525072e-f5b3-4c68-800c-6292353b55bd-trusted-ca\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.342189 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.342109    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e525072e-f5b3-4c68-800c-6292353b55bd-installation-pull-secrets\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.343208 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.343183    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e59d69f3-69b5-4d85-8a54-f9def9627f85-additional-service-ca\") pod \"cluster-proxy-proxy-agent-77bd48578d-jjgt9\" (UID: \"e59d69f3-69b5-4d85-8a54-f9def9627f85\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"
Jul 28 22:05:25.344906 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.344870    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e59d69f3-69b5-4d85-8a54-f9def9627f85-ca\") pod \"cluster-proxy-proxy-agent-77bd48578d-jjgt9\" (UID: \"e59d69f3-69b5-4d85-8a54-f9def9627f85\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"
Jul 28 22:05:25.345024 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.344976    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6e311d08-0827-46f7-8382-9d8ddaaa73de-klusterlet-config\") pod \"klusterlet-addon-workmgr-5c9b9dc685-ftgft\" (UID: \"6e311d08-0827-46f7-8382-9d8ddaaa73de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c9b9dc685-ftgft"
Jul 28 22:05:25.345615 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.345490    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5e7aa6b1-aa47-4a47-8398-5d9cd4b9126b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-bd9bc875b-pb4kg\" (UID: \"5e7aa6b1-aa47-4a47-8398-5d9cd4b9126b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bd9bc875b-pb4kg"
Jul 28 22:05:25.346353 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.345956    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e59d69f3-69b5-4d85-8a54-f9def9627f85-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-77bd48578d-jjgt9\" (UID: \"e59d69f3-69b5-4d85-8a54-f9def9627f85\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"
Jul 28 22:05:25.346353 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.346185    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e59d69f3-69b5-4d85-8a54-f9def9627f85-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-77bd48578d-jjgt9\" (UID: \"e59d69f3-69b5-4d85-8a54-f9def9627f85\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"
Jul 28 22:05:25.347061 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.347039    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e59d69f3-69b5-4d85-8a54-f9def9627f85-hub\") pod \"cluster-proxy-proxy-agent-77bd48578d-jjgt9\" (UID: \"e59d69f3-69b5-4d85-8a54-f9def9627f85\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"
Jul 28 22:05:25.356039 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.356003    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjb9n\" (UniqueName: \"kubernetes.io/projected/e59d69f3-69b5-4d85-8a54-f9def9627f85-kube-api-access-rjb9n\") pod \"cluster-proxy-proxy-agent-77bd48578d-jjgt9\" (UID: \"e59d69f3-69b5-4d85-8a54-f9def9627f85\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"
Jul 28 22:05:25.358737 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.358712    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgw9c\" (UniqueName: \"kubernetes.io/projected/5e7aa6b1-aa47-4a47-8398-5d9cd4b9126b-kube-api-access-jgw9c\") pod \"managed-serviceaccount-addon-agent-bd9bc875b-pb4kg\" (UID: \"5e7aa6b1-aa47-4a47-8398-5d9cd4b9126b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bd9bc875b-pb4kg"
Jul 28 22:05:25.358836 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.358770    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dscx\" (UniqueName: \"kubernetes.io/projected/6e311d08-0827-46f7-8382-9d8ddaaa73de-kube-api-access-4dscx\") pod \"klusterlet-addon-workmgr-5c9b9dc685-ftgft\" (UID: \"6e311d08-0827-46f7-8382-9d8ddaaa73de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c9b9dc685-ftgft"
Jul 28 22:05:25.442846 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.442810    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjjzn\" (UniqueName: \"kubernetes.io/projected/e525072e-f5b3-4c68-800c-6292353b55bd-kube-api-access-vjjzn\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.443007 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.442862    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e525072e-f5b3-4c68-800c-6292353b55bd-registry-tls\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.443007 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.442950    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e525072e-f5b3-4c68-800c-6292353b55bd-ca-trust-extracted\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.443007 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.442969    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e525072e-f5b3-4c68-800c-6292353b55bd-registry-certificates\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.443127 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.443090    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e525072e-f5b3-4c68-800c-6292353b55bd-bound-sa-token\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.443259 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.443173    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e525072e-f5b3-4c68-800c-6292353b55bd-image-registry-private-configuration\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.443259 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.443223    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e525072e-f5b3-4c68-800c-6292353b55bd-trusted-ca\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.443390 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.443265    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e525072e-f5b3-4c68-800c-6292353b55bd-installation-pull-secrets\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.443390 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.443371    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e525072e-f5b3-4c68-800c-6292353b55bd-ca-trust-extracted\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.443921 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.443886    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e525072e-f5b3-4c68-800c-6292353b55bd-registry-certificates\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.444578 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.444559    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e525072e-f5b3-4c68-800c-6292353b55bd-trusted-ca\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.445493 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.445473    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bd9bc875b-pb4kg"
Jul 28 22:05:25.445837 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.445809    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e525072e-f5b3-4c68-800c-6292353b55bd-image-registry-private-configuration\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.445935 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.445923    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e525072e-f5b3-4c68-800c-6292353b55bd-installation-pull-secrets\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.446111 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.446091    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e525072e-f5b3-4c68-800c-6292353b55bd-registry-tls\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.453774 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.453749    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c9b9dc685-ftgft"
Jul 28 22:05:25.454474 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.454454    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e525072e-f5b3-4c68-800c-6292353b55bd-bound-sa-token\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.454811 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.454788    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjjzn\" (UniqueName: \"kubernetes.io/projected/e525072e-f5b3-4c68-800c-6292353b55bd-kube-api-access-vjjzn\") pod \"image-registry-74b5c9c64-f7qff\" (UID: \"e525072e-f5b3-4c68-800c-6292353b55bd\") " pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:25.459604 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.459583    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"
Jul 28 22:05:25.543520 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:25.543481    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:26.323043 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:26.323008    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c9b9dc685-ftgft"]
Jul 28 22:05:26.331176 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:05:26.331147    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e311d08_0827_46f7_8382_9d8ddaaa73de.slice/crio-48ab27e0fd855198fcdc4a5be430ec316a772b46d6ff50626e375213efa7f647 WatchSource:0}: Error finding container 48ab27e0fd855198fcdc4a5be430ec316a772b46d6ff50626e375213efa7f647: Status 404 returned error can't find the container with id 48ab27e0fd855198fcdc4a5be430ec316a772b46d6ff50626e375213efa7f647
Jul 28 22:05:26.336477 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:26.336421    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9"]
Jul 28 22:05:26.339878 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:05:26.339847    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode59d69f3_69b5_4d85_8a54_f9def9627f85.slice/crio-813c6344b0c11ce97378e06ae692feb0f1e2593e9a8e891d513feecc99d7d9b8 WatchSource:0}: Error finding container 813c6344b0c11ce97378e06ae692feb0f1e2593e9a8e891d513feecc99d7d9b8: Status 404 returned error can't find the container with id 813c6344b0c11ce97378e06ae692feb0f1e2593e9a8e891d513feecc99d7d9b8
Jul 28 22:05:26.544413 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:26.544383    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bd9bc875b-pb4kg"]
Jul 28 22:05:26.547120 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:26.547023    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74b5c9c64-f7qff"]
Jul 28 22:05:26.547951 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:05:26.547926    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e7aa6b1_aa47_4a47_8398_5d9cd4b9126b.slice/crio-fa4fb0171d86e9a4713cea172cd53593ede1e035679c01950487f0948b864733 WatchSource:0}: Error finding container fa4fb0171d86e9a4713cea172cd53593ede1e035679c01950487f0948b864733: Status 404 returned error can't find the container with id fa4fb0171d86e9a4713cea172cd53593ede1e035679c01950487f0948b864733
Jul 28 22:05:26.550416 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:05:26.550391    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode525072e_f5b3_4c68_800c_6292353b55bd.slice/crio-7fc2d0eeddb5dd0e748885feb0de6d9baee0449b3bebcac939bcd06ba2a591bf WatchSource:0}: Error finding container 7fc2d0eeddb5dd0e748885feb0de6d9baee0449b3bebcac939bcd06ba2a591bf: Status 404 returned error can't find the container with id 7fc2d0eeddb5dd0e748885feb0de6d9baee0449b3bebcac939bcd06ba2a591bf
Jul 28 22:05:26.751394 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:26.751364    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hqgs4"
Jul 28 22:05:26.775320 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:26.775284    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bd9bc875b-pb4kg" event={"ID":"5e7aa6b1-aa47-4a47-8398-5d9cd4b9126b","Type":"ContainerStarted","Data":"fa4fb0171d86e9a4713cea172cd53593ede1e035679c01950487f0948b864733"}
Jul 28 22:05:26.776752 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:26.776728    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9" event={"ID":"e59d69f3-69b5-4d85-8a54-f9def9627f85","Type":"ContainerStarted","Data":"813c6344b0c11ce97378e06ae692feb0f1e2593e9a8e891d513feecc99d7d9b8"}
Jul 28 22:05:26.778675 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:26.778471    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74b5c9c64-f7qff" event={"ID":"e525072e-f5b3-4c68-800c-6292353b55bd","Type":"ContainerStarted","Data":"c70dc9205cb3f99169076a91990773fb2968a607c636a92836552d269fa80aa0"}
Jul 28 22:05:26.778675 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:26.778518    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74b5c9c64-f7qff" event={"ID":"e525072e-f5b3-4c68-800c-6292353b55bd","Type":"ContainerStarted","Data":"7fc2d0eeddb5dd0e748885feb0de6d9baee0449b3bebcac939bcd06ba2a591bf"}
Jul 28 22:05:26.778675 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:26.778567    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:26.780360 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:26.780330    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lhf6r" event={"ID":"b5d9fd68-fbef-4b56-ac86-4b26358778e7","Type":"ContainerStarted","Data":"1bd1e90fd454c023d1f2c83772172906c9056bb60a3fd78cbd4053b28a47af28"}
Jul 28 22:05:26.781529 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:26.781506    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c9b9dc685-ftgft" event={"ID":"6e311d08-0827-46f7-8382-9d8ddaaa73de","Type":"ContainerStarted","Data":"48ab27e0fd855198fcdc4a5be430ec316a772b46d6ff50626e375213efa7f647"}
Jul 28 22:05:26.800320 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:26.800193    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-74b5c9c64-f7qff" podStartSLOduration=1.8001721339999999 podStartE2EDuration="1.800172134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:05:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:26.798666848 +0000 UTC m=+55.782809470" watchObservedRunningTime="2026-07-28 22:05:26.800172134 +0000 UTC m=+55.784314755"
Jul 28 22:05:30.691265 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:30.691218    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jrh49"
Jul 28 22:05:34.814910 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:34.814813    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bd9bc875b-pb4kg" event={"ID":"5e7aa6b1-aa47-4a47-8398-5d9cd4b9126b","Type":"ContainerStarted","Data":"50f64d98c4f1472b05adc3ce2db1467739fa3ec54a2c969cf05bcdfbbfa4ebce"}
Jul 28 22:05:34.817013 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:34.816987    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9" event={"ID":"e59d69f3-69b5-4d85-8a54-f9def9627f85","Type":"ContainerStarted","Data":"4fbeb3e6f96180c97b4797d024333f1e733cf8c2ba107c0cb4d2d4f6e3bae5df"}
Jul 28 22:05:34.817143 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:34.817027    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9" event={"ID":"e59d69f3-69b5-4d85-8a54-f9def9627f85","Type":"ContainerStarted","Data":"ff79ec0228deab98403bd4099ce6693c6e93791fa25c41a4bac297363e773066"}
Jul 28 22:05:34.817143 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:34.817045    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9" event={"ID":"e59d69f3-69b5-4d85-8a54-f9def9627f85","Type":"ContainerStarted","Data":"be6a0792f620671b958d404fa6f2f298be995ec9bc9e5a4c7f51acb4cac1ba0f"}
Jul 28 22:05:34.818870 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:34.818847    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lhf6r" event={"ID":"b5d9fd68-fbef-4b56-ac86-4b26358778e7","Type":"ContainerStarted","Data":"964bc64b0e6f26a07cd708501858265d91858be4d0d87994dd39fd0ce12fbacf"}
Jul 28 22:05:34.820271 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:34.820246    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c9b9dc685-ftgft" event={"ID":"6e311d08-0827-46f7-8382-9d8ddaaa73de","Type":"ContainerStarted","Data":"7af7b38c0a2b54d2ecad79bf65b5604b821e1f02ec5f30f5b02b35b6bcd79d07"}
Jul 28 22:05:34.820449 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:34.820433    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c9b9dc685-ftgft"
Jul 28 22:05:34.822180 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:34.822155    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c9b9dc685-ftgft"
Jul 28 22:05:34.831505 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:34.831467    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bd9bc875b-pb4kg" podStartSLOduration=1.872758641 podStartE2EDuration="9.831454635s" totalImagesPullingTime="7.958695994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:34.83127505 +0000 UTC m=+63.815417671" watchObservedRunningTime="2026-07-28 22:05:34.831454635 +0000 UTC m=+63.815597256"
Jul 28 22:05:34.847805 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:34.847754    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c9b9dc685-ftgft" podStartSLOduration=1.668809296 podStartE2EDuration="9.847738291s" totalImagesPullingTime="8.178928995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:34.846609545 +0000 UTC m=+63.830752165" watchObservedRunningTime="2026-07-28 22:05:34.847738291 +0000 UTC m=+63.831880910"
Jul 28 22:05:34.865869 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:34.865822    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77bd48578d-jjgt9" podStartSLOduration=1.652301383 podStartE2EDuration="9.865806012s" totalImagesPullingTime="8.213504629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:34.865411523 +0000 UTC m=+63.849554143" watchObservedRunningTime="2026-07-28 22:05:34.865806012 +0000 UTC m=+63.849948609"
Jul 28 22:05:35.074456 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:35.074356    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:36.238151 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:36.238094    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret\") pod \"global-pull-secret-syncer-qw7gj\" (UID: \"6bbf80a3-a5e2-49a5-ad56-a21747cc4a16\") " pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:05:36.238630 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:36.238170    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs\") pod \"network-metrics-daemon-dvqcn\" (UID: \"f738f750-e0f4-49e7-b7e6-0c151e664ce9\") " pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:05:36.240312 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:36.240288    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 22:05:36.240700 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:36.240683    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 22:05:36.251644 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:36.251617    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6bbf80a3-a5e2-49a5-ad56-a21747cc4a16-original-pull-secret\") pod \"global-pull-secret-syncer-qw7gj\" (UID: \"6bbf80a3-a5e2-49a5-ad56-a21747cc4a16\") " pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:05:36.251774 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:36.251617    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f738f750-e0f4-49e7-b7e6-0c151e664ce9-metrics-certs\") pod \"network-metrics-daemon-dvqcn\" (UID: \"f738f750-e0f4-49e7-b7e6-0c151e664ce9\") " pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:05:36.338959 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:36.338920    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dh6q\" (UniqueName: \"kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q\") pod \"network-check-target-np5j9\" (UID: \"8497aa3c-a69a-4e64-a1f3-306cd512ae81\") " pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:05:36.341176 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:36.341159    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 22:05:36.351219 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:36.351199    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 22:05:36.362907 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:36.362885    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dh6q\" (UniqueName: \"kubernetes.io/projected/8497aa3c-a69a-4e64-a1f3-306cd512ae81-kube-api-access-6dh6q\") pod \"network-check-target-np5j9\" (UID: \"8497aa3c-a69a-4e64-a1f3-306cd512ae81\") " pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:05:36.403420 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:36.403384    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sm6xq\""
Jul 28 22:05:36.411837 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:36.411812    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvqcn"
Jul 28 22:05:36.432454 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:36.432428    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bnqzv\""
Jul 28 22:05:36.437857 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:36.437831    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw7gj"
Jul 28 22:05:36.441014 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:36.440998    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:05:36.611434 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:36.611398    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dvqcn"]
Jul 28 22:05:36.647910 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:36.647877    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qw7gj"]
Jul 28 22:05:38.218199 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:05:38.218153    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bbf80a3_a5e2_49a5_ad56_a21747cc4a16.slice/crio-aad08f3ac51a7881d901444e4db4ae27ec0a2d53492ddc47df08b3611af6bbeb WatchSource:0}: Error finding container aad08f3ac51a7881d901444e4db4ae27ec0a2d53492ddc47df08b3611af6bbeb: Status 404 returned error can't find the container with id aad08f3ac51a7881d901444e4db4ae27ec0a2d53492ddc47df08b3611af6bbeb
Jul 28 22:05:38.218638 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:05:38.218540    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf738f750_e0f4_49e7_b7e6_0c151e664ce9.slice/crio-eed2f2ace0d0afd8e30c7364b64fa66ab092c79fb97a7d02d7ab176dde3a2e6d WatchSource:0}: Error finding container eed2f2ace0d0afd8e30c7364b64fa66ab092c79fb97a7d02d7ab176dde3a2e6d: Status 404 returned error can't find the container with id eed2f2ace0d0afd8e30c7364b64fa66ab092c79fb97a7d02d7ab176dde3a2e6d
Jul 28 22:05:38.344499 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:38.344471    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-np5j9"]
Jul 28 22:05:38.347366 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:05:38.347335    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8497aa3c_a69a_4e64_a1f3_306cd512ae81.slice/crio-c5700965cb9282b745a3d4abd10c1b9ffa856f0cd950b2c4391fd1276261857d WatchSource:0}: Error finding container c5700965cb9282b745a3d4abd10c1b9ffa856f0cd950b2c4391fd1276261857d: Status 404 returned error can't find the container with id c5700965cb9282b745a3d4abd10c1b9ffa856f0cd950b2c4391fd1276261857d
Jul 28 22:05:38.837941 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:38.837844    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qw7gj" event={"ID":"6bbf80a3-a5e2-49a5-ad56-a21747cc4a16","Type":"ContainerStarted","Data":"aad08f3ac51a7881d901444e4db4ae27ec0a2d53492ddc47df08b3611af6bbeb"}
Jul 28 22:05:38.841400 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:38.841323    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lhf6r" event={"ID":"b5d9fd68-fbef-4b56-ac86-4b26358778e7","Type":"ContainerStarted","Data":"cad9321d06385d22c9aa2a505f47ea26866828121b97bf391b3793d46ccee665"}
Jul 28 22:05:38.843634 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:38.843558    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-np5j9" event={"ID":"8497aa3c-a69a-4e64-a1f3-306cd512ae81","Type":"ContainerStarted","Data":"c5700965cb9282b745a3d4abd10c1b9ffa856f0cd950b2c4391fd1276261857d"}
Jul 28 22:05:38.845329 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:38.845301    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dvqcn" event={"ID":"f738f750-e0f4-49e7-b7e6-0c151e664ce9","Type":"ContainerStarted","Data":"eed2f2ace0d0afd8e30c7364b64fa66ab092c79fb97a7d02d7ab176dde3a2e6d"}
Jul 28 22:05:38.858968 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:38.858902    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lhf6r" podStartSLOduration=17.75630827 podStartE2EDuration="34.858885442s" totalImagesPullingTime="17.102577172s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:05:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:38.858011961 +0000 UTC m=+67.842154584" watchObservedRunningTime="2026-07-28 22:05:38.858885442 +0000 UTC m=+67.843028062"
Jul 28 22:05:39.852889 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:39.852833    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dvqcn" event={"ID":"f738f750-e0f4-49e7-b7e6-0c151e664ce9","Type":"ContainerStarted","Data":"038e9846f57249758c33b2ba5e12c7f64a2379e776d41ec2de028c657fa5f13c"}
Jul 28 22:05:39.853401 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:39.852920    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dvqcn" event={"ID":"f738f750-e0f4-49e7-b7e6-0c151e664ce9","Type":"ContainerStarted","Data":"99250bfac7fee0d0616cea7e1f79bd1e835b5c478ea3d4c8d3b59d8b36335001"}
Jul 28 22:05:39.872185 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:39.871868    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dvqcn" podStartSLOduration=67.69879707 podStartE2EDuration="1m8.871847842s" totalImagesPullingTime="1.173050772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:39.86991767 +0000 UTC m=+68.854060316" watchObservedRunningTime="2026-07-28 22:05:39.871847842 +0000 UTC m=+68.855990464"
Jul 28 22:05:43.870046 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:43.870001    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qw7gj" event={"ID":"6bbf80a3-a5e2-49a5-ad56-a21747cc4a16","Type":"ContainerStarted","Data":"e750c90f33a3d5c10eb60cc14820eeff0cd95a666d5dbb5a05c02a9b360183b0"}
Jul 28 22:05:43.871566 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:43.871531    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-np5j9" event={"ID":"8497aa3c-a69a-4e64-a1f3-306cd512ae81","Type":"ContainerStarted","Data":"b2a166d637c3ebde79ea61e86498cdb5ea3f008b25c337d5ac2c35daef01e3be"}
Jul 28 22:05:43.871680 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:43.871600    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:05:43.884383 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:43.884334    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qw7gj" podStartSLOduration=68.195021277 podStartE2EDuration="1m12.884319635s" totalImagesPullingTime="4.689298358s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:43.883165039 +0000 UTC m=+72.867307658" watchObservedRunningTime="2026-07-28 22:05:43.884319635 +0000 UTC m=+72.868462254"
Jul 28 22:05:43.897967 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:43.897922    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-np5j9" podStartSLOduration=68.342487275 podStartE2EDuration="1m12.897907069s" totalImagesPullingTime="4.555419794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:43.897604963 +0000 UTC m=+72.881747583" watchObservedRunningTime="2026-07-28 22:05:43.897907069 +0000 UTC m=+72.882049689"
Jul 28 22:05:47.791288 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:47.791257    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-74b5c9c64-f7qff"
Jul 28 22:05:50.090925 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.090877    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-66bff7f44f-mznc6" podUID="bd70751c-6f8d-4c3e-9602-cfa827b322f3" containerName="registry" containerID="cri-o://4f4f77a506aa9d46a89ee4ccba88d4fd77e9542ac94b0c42a421c1774ebb7a30" gracePeriod=30
Jul 28 22:05:50.340625 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.340598    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:50.470460 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.470424    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-tls\") pod \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") "
Jul 28 22:05:50.470612 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.470466    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6gjk\" (UniqueName: \"kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-kube-api-access-m6gjk\") pod \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") "
Jul 28 22:05:50.470612 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.470484    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-bound-sa-token\") pod \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") "
Jul 28 22:05:50.470612 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.470531    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-certificates\") pod \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") "
Jul 28 22:05:50.470740 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.470711    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bd70751c-6f8d-4c3e-9602-cfa827b322f3-ca-trust-extracted\") pod \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") "
Jul 28 22:05:50.470884 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.470768    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bd70751c-6f8d-4c3e-9602-cfa827b322f3-image-registry-private-configuration\") pod \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") "
Jul 28 22:05:50.470884 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.470843    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bd70751c-6f8d-4c3e-9602-cfa827b322f3-installation-pull-secrets\") pod \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") "
Jul 28 22:05:50.470884 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.470874    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bd70751c-6f8d-4c3e-9602-cfa827b322f3-trusted-ca\") pod \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\" (UID: \"bd70751c-6f8d-4c3e-9602-cfa827b322f3\") "
Jul 28 22:05:50.471058 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.471021    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "bd70751c-6f8d-4c3e-9602-cfa827b322f3" (UID: "bd70751c-6f8d-4c3e-9602-cfa827b322f3"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:05:50.471139 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.471118    2574 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-certificates\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:05:50.471449 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.471422    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd70751c-6f8d-4c3e-9602-cfa827b322f3-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "bd70751c-6f8d-4c3e-9602-cfa827b322f3" (UID: "bd70751c-6f8d-4c3e-9602-cfa827b322f3"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:05:50.473317 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.473267    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "bd70751c-6f8d-4c3e-9602-cfa827b322f3" (UID: "bd70751c-6f8d-4c3e-9602-cfa827b322f3"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:05:50.473317 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.473295    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd70751c-6f8d-4c3e-9602-cfa827b322f3-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "bd70751c-6f8d-4c3e-9602-cfa827b322f3" (UID: "bd70751c-6f8d-4c3e-9602-cfa827b322f3"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:05:50.473449 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.473334    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bd70751c-6f8d-4c3e-9602-cfa827b322f3" (UID: "bd70751c-6f8d-4c3e-9602-cfa827b322f3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:05:50.473449 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.473411    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-kube-api-access-m6gjk" (OuterVolumeSpecName: "kube-api-access-m6gjk") pod "bd70751c-6f8d-4c3e-9602-cfa827b322f3" (UID: "bd70751c-6f8d-4c3e-9602-cfa827b322f3"). InnerVolumeSpecName "kube-api-access-m6gjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:05:50.473531 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.473494    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd70751c-6f8d-4c3e-9602-cfa827b322f3-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "bd70751c-6f8d-4c3e-9602-cfa827b322f3" (UID: "bd70751c-6f8d-4c3e-9602-cfa827b322f3"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:05:50.479280 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.479226    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd70751c-6f8d-4c3e-9602-cfa827b322f3-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "bd70751c-6f8d-4c3e-9602-cfa827b322f3" (UID: "bd70751c-6f8d-4c3e-9602-cfa827b322f3"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:05:50.571645 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.571607    2574 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-registry-tls\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:05:50.571645 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.571634    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6gjk\" (UniqueName: \"kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-kube-api-access-m6gjk\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:05:50.571645 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.571644    2574 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bd70751c-6f8d-4c3e-9602-cfa827b322f3-bound-sa-token\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:05:50.571645 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.571654    2574 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bd70751c-6f8d-4c3e-9602-cfa827b322f3-ca-trust-extracted\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:05:50.571889 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.571663    2574 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bd70751c-6f8d-4c3e-9602-cfa827b322f3-image-registry-private-configuration\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:05:50.571889 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.571673    2574 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bd70751c-6f8d-4c3e-9602-cfa827b322f3-installation-pull-secrets\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:05:50.571889 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.571682    2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bd70751c-6f8d-4c3e-9602-cfa827b322f3-trusted-ca\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:05:50.654204 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.654165    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-b5d92"]
Jul 28 22:05:50.654435 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.654421    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd70751c-6f8d-4c3e-9602-cfa827b322f3" containerName="registry"
Jul 28 22:05:50.654477 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.654437    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd70751c-6f8d-4c3e-9602-cfa827b322f3" containerName="registry"
Jul 28 22:05:50.654515 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.654491    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd70751c-6f8d-4c3e-9602-cfa827b322f3" containerName="registry"
Jul 28 22:05:50.660120 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.660103    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.662207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.662182    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 22:05:50.662358 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.662290    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 22:05:50.662358 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.662347    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 22:05:50.662556 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.662538    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gsb4z\""
Jul 28 22:05:50.662625 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.662557    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 22:05:50.662625 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.662567    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 22:05:50.662802 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.662789    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 22:05:50.773412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.773318    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-node-exporter-accelerators-collector-config\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.773412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.773360    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h469\" (UniqueName: \"kubernetes.io/projected/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-kube-api-access-6h469\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.773412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.773376    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-node-exporter-tls\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.773412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.773390    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-metrics-client-ca\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.773683 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.773439    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-node-exporter-textfile\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.773683 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.773499    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.773683 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.773534    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-root\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.773683 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.773557    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-sys\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.773683 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.773579    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-node-exporter-wtmp\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.874435 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.874398    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.874435 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.874442    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-root\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.874693 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.874462    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-sys\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.874693 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.874477    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-node-exporter-wtmp\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.874693 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.874514    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-node-exporter-accelerators-collector-config\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.874693 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.874533    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h469\" (UniqueName: \"kubernetes.io/projected/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-kube-api-access-6h469\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.874693 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.874546    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-node-exporter-tls\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.874693 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.874545    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-sys\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.874693 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.874564    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-metrics-client-ca\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.874693 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.874544    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-root\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.874693 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.874585    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-node-exporter-textfile\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.875159 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:50.874706    2574 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 22:05:50.875159 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.874744    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-node-exporter-wtmp\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.875159 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:50.874784    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-node-exporter-tls podName:0e15a45f-7342-41b6-a8d3-3d3b7e2945ae nodeName:}" failed. No retries permitted until 2026-07-28 22:05:51.374762842 +0000 UTC m=+80.358905460 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-node-exporter-tls") pod "node-exporter-b5d92" (UID: "0e15a45f-7342-41b6-a8d3-3d3b7e2945ae") : secret "node-exporter-tls" not found
Jul 28 22:05:50.875159 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.874958    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-node-exporter-textfile\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.875347 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.875172    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-metrics-client-ca\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.875347 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.875194    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-node-exporter-accelerators-collector-config\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.876835 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.876818    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.882552 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.882531    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h469\" (UniqueName: \"kubernetes.io/projected/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-kube-api-access-6h469\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:50.894113 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.894091    2574 generic.go:356] "Generic (PLEG): container finished" podID="bd70751c-6f8d-4c3e-9602-cfa827b322f3" containerID="4f4f77a506aa9d46a89ee4ccba88d4fd77e9542ac94b0c42a421c1774ebb7a30" exitCode=0
Jul 28 22:05:50.894224 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.894129    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66bff7f44f-mznc6" event={"ID":"bd70751c-6f8d-4c3e-9602-cfa827b322f3","Type":"ContainerDied","Data":"4f4f77a506aa9d46a89ee4ccba88d4fd77e9542ac94b0c42a421c1774ebb7a30"}
Jul 28 22:05:50.894224 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.894154    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66bff7f44f-mznc6"
Jul 28 22:05:50.894224 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.894189    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66bff7f44f-mznc6" event={"ID":"bd70751c-6f8d-4c3e-9602-cfa827b322f3","Type":"ContainerDied","Data":"9dcb39eb1079da7daa36e0afd07606d59018978191a087b4f4e074ee719b5a7c"}
Jul 28 22:05:50.894224 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.894217    2574 scope.go:117] "RemoveContainer" containerID="4f4f77a506aa9d46a89ee4ccba88d4fd77e9542ac94b0c42a421c1774ebb7a30"
Jul 28 22:05:50.902454 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.902438    2574 scope.go:117] "RemoveContainer" containerID="4f4f77a506aa9d46a89ee4ccba88d4fd77e9542ac94b0c42a421c1774ebb7a30"
Jul 28 22:05:50.902737 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:05:50.902717    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f4f77a506aa9d46a89ee4ccba88d4fd77e9542ac94b0c42a421c1774ebb7a30\": container with ID starting with 4f4f77a506aa9d46a89ee4ccba88d4fd77e9542ac94b0c42a421c1774ebb7a30 not found: ID does not exist" containerID="4f4f77a506aa9d46a89ee4ccba88d4fd77e9542ac94b0c42a421c1774ebb7a30"
Jul 28 22:05:50.902795 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.902745    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f4f77a506aa9d46a89ee4ccba88d4fd77e9542ac94b0c42a421c1774ebb7a30"} err="failed to get container status \"4f4f77a506aa9d46a89ee4ccba88d4fd77e9542ac94b0c42a421c1774ebb7a30\": rpc error: code = NotFound desc = could not find container \"4f4f77a506aa9d46a89ee4ccba88d4fd77e9542ac94b0c42a421c1774ebb7a30\": container with ID starting with 4f4f77a506aa9d46a89ee4ccba88d4fd77e9542ac94b0c42a421c1774ebb7a30 not found: ID does not exist"
Jul 28 22:05:50.918517 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.918490    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66bff7f44f-mznc6"]
Jul 28 22:05:50.922223 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:50.922202    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-66bff7f44f-mznc6"]
Jul 28 22:05:51.379125 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:51.379087    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-node-exporter-tls\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:51.381477 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:51.381455    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0e15a45f-7342-41b6-a8d3-3d3b7e2945ae-node-exporter-tls\") pod \"node-exporter-b5d92\" (UID: \"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae\") " pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:51.491743 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:51.491711    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd70751c-6f8d-4c3e-9602-cfa827b322f3" path="/var/lib/kubelet/pods/bd70751c-6f8d-4c3e-9602-cfa827b322f3/volumes"
Jul 28 22:05:51.568928 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:51.568901    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b5d92"
Jul 28 22:05:51.582596 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:05:51.582568    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e15a45f_7342_41b6_a8d3_3d3b7e2945ae.slice/crio-65fd634ab05695967689bf99a367e3bf918c0b5e31eb6dddf67f3c9ad70c1199 WatchSource:0}: Error finding container 65fd634ab05695967689bf99a367e3bf918c0b5e31eb6dddf67f3c9ad70c1199: Status 404 returned error can't find the container with id 65fd634ab05695967689bf99a367e3bf918c0b5e31eb6dddf67f3c9ad70c1199
Jul 28 22:05:51.899004 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:51.898960    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b5d92" event={"ID":"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae","Type":"ContainerStarted","Data":"65fd634ab05695967689bf99a367e3bf918c0b5e31eb6dddf67f3c9ad70c1199"}
Jul 28 22:05:52.905375 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:52.905345    2574 generic.go:356] "Generic (PLEG): container finished" podID="0e15a45f-7342-41b6-a8d3-3d3b7e2945ae" containerID="03bf06b533cef9af6d0a401e30700740e5d3a12b52162adaba8a7a86a59dcf0d" exitCode=0
Jul 28 22:05:52.905745 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:52.905423    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b5d92" event={"ID":"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae","Type":"ContainerDied","Data":"03bf06b533cef9af6d0a401e30700740e5d3a12b52162adaba8a7a86a59dcf0d"}
Jul 28 22:05:53.911374 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:53.911331    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b5d92" event={"ID":"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae","Type":"ContainerStarted","Data":"46229224f876bacd5245b4289ef4534d505be2b72c7f12621d1a2ac25925072a"}
Jul 28 22:05:53.911798 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:53.911401    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b5d92" event={"ID":"0e15a45f-7342-41b6-a8d3-3d3b7e2945ae","Type":"ContainerStarted","Data":"bf1b81914934975e8be702cf29599c99169e51dc791ae6f2c2bd1ef4f36a309f"}
Jul 28 22:05:53.928900 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:05:53.928845    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-b5d92" podStartSLOduration=2.738221582 podStartE2EDuration="3.928830904s" totalImagesPullingTime="1.127306492s" totalInitContainerRuntime="63.30283ms" isStatefulPod=true podCreationTimestamp="2026-07-28 22:05:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:53.927766707 +0000 UTC m=+82.911909330" watchObservedRunningTime="2026-07-28 22:05:53.928830904 +0000 UTC m=+82.912973533"
Jul 28 22:06:14.877025 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:06:14.876992    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-np5j9"
Jul 28 22:09:31.335785 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:31.335758    2574 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 22:09:35.593601 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.593567    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v"]
Jul 28 22:09:35.595548 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.595530    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v"
Jul 28 22:09:35.597569 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.597546    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 22:09:35.598121 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.598096    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gbw8q\""
Jul 28 22:09:35.598378 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.598363    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 22:09:35.598643 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.598623    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 22:09:35.598732 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.598717    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 22:09:35.613108 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.613084    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v"]
Jul 28 22:09:35.769595 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.769558    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b03a58fd-5c5b-44fd-b6c2-cc9a6a455178-tmp\") pod \"opendatahub-operator-controller-manager-d45996775-fzq6v\" (UID: \"b03a58fd-5c5b-44fd-b6c2-cc9a6a455178\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v"
Jul 28 22:09:35.769783 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.769606    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46vfc\" (UniqueName: \"kubernetes.io/projected/b03a58fd-5c5b-44fd-b6c2-cc9a6a455178-kube-api-access-46vfc\") pod \"opendatahub-operator-controller-manager-d45996775-fzq6v\" (UID: \"b03a58fd-5c5b-44fd-b6c2-cc9a6a455178\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v"
Jul 28 22:09:35.769783 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.769631    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b03a58fd-5c5b-44fd-b6c2-cc9a6a455178-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d45996775-fzq6v\" (UID: \"b03a58fd-5c5b-44fd-b6c2-cc9a6a455178\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v"
Jul 28 22:09:35.769783 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.769649    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b03a58fd-5c5b-44fd-b6c2-cc9a6a455178-manifests\") pod \"opendatahub-operator-controller-manager-d45996775-fzq6v\" (UID: \"b03a58fd-5c5b-44fd-b6c2-cc9a6a455178\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v"
Jul 28 22:09:35.769783 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.769733    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b03a58fd-5c5b-44fd-b6c2-cc9a6a455178-webhook-cert\") pod \"opendatahub-operator-controller-manager-d45996775-fzq6v\" (UID: \"b03a58fd-5c5b-44fd-b6c2-cc9a6a455178\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v"
Jul 28 22:09:35.870777 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.870679    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b03a58fd-5c5b-44fd-b6c2-cc9a6a455178-webhook-cert\") pod \"opendatahub-operator-controller-manager-d45996775-fzq6v\" (UID: \"b03a58fd-5c5b-44fd-b6c2-cc9a6a455178\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v"
Jul 28 22:09:35.870777 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.870743    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b03a58fd-5c5b-44fd-b6c2-cc9a6a455178-tmp\") pod \"opendatahub-operator-controller-manager-d45996775-fzq6v\" (UID: \"b03a58fd-5c5b-44fd-b6c2-cc9a6a455178\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v"
Jul 28 22:09:35.871011 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.870779    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46vfc\" (UniqueName: \"kubernetes.io/projected/b03a58fd-5c5b-44fd-b6c2-cc9a6a455178-kube-api-access-46vfc\") pod \"opendatahub-operator-controller-manager-d45996775-fzq6v\" (UID: \"b03a58fd-5c5b-44fd-b6c2-cc9a6a455178\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v"
Jul 28 22:09:35.871011 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.870809    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b03a58fd-5c5b-44fd-b6c2-cc9a6a455178-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d45996775-fzq6v\" (UID: \"b03a58fd-5c5b-44fd-b6c2-cc9a6a455178\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v"
Jul 28 22:09:35.871011 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.870836    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b03a58fd-5c5b-44fd-b6c2-cc9a6a455178-manifests\") pod \"opendatahub-operator-controller-manager-d45996775-fzq6v\" (UID: \"b03a58fd-5c5b-44fd-b6c2-cc9a6a455178\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v"
Jul 28 22:09:35.871333 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.871301    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b03a58fd-5c5b-44fd-b6c2-cc9a6a455178-manifests\") pod \"opendatahub-operator-controller-manager-d45996775-fzq6v\" (UID: \"b03a58fd-5c5b-44fd-b6c2-cc9a6a455178\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v"
Jul 28 22:09:35.873139 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.873110    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b03a58fd-5c5b-44fd-b6c2-cc9a6a455178-tmp\") pod \"opendatahub-operator-controller-manager-d45996775-fzq6v\" (UID: \"b03a58fd-5c5b-44fd-b6c2-cc9a6a455178\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v"
Jul 28 22:09:35.873320 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.873300    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b03a58fd-5c5b-44fd-b6c2-cc9a6a455178-webhook-cert\") pod \"opendatahub-operator-controller-manager-d45996775-fzq6v\" (UID: \"b03a58fd-5c5b-44fd-b6c2-cc9a6a455178\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v"
Jul 28 22:09:35.873374 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.873297    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b03a58fd-5c5b-44fd-b6c2-cc9a6a455178-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d45996775-fzq6v\" (UID: \"b03a58fd-5c5b-44fd-b6c2-cc9a6a455178\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v"
Jul 28 22:09:35.877796 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.877769    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46vfc\" (UniqueName: \"kubernetes.io/projected/b03a58fd-5c5b-44fd-b6c2-cc9a6a455178-kube-api-access-46vfc\") pod \"opendatahub-operator-controller-manager-d45996775-fzq6v\" (UID: \"b03a58fd-5c5b-44fd-b6c2-cc9a6a455178\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v"
Jul 28 22:09:35.904847 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:35.904820    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v"
Jul 28 22:09:36.037564 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:36.037531    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v"]
Jul 28 22:09:36.041749 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:09:36.041721    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb03a58fd_5c5b_44fd_b6c2_cc9a6a455178.slice/crio-2ca813ded920368d0d5fdf4c79beb2d6db8429d617723880a37cfe7fbfef4d8d WatchSource:0}: Error finding container 2ca813ded920368d0d5fdf4c79beb2d6db8429d617723880a37cfe7fbfef4d8d: Status 404 returned error can't find the container with id 2ca813ded920368d0d5fdf4c79beb2d6db8429d617723880a37cfe7fbfef4d8d
Jul 28 22:09:36.043589 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:36.043570    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 22:09:36.621052 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:36.620976    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v" event={"ID":"b03a58fd-5c5b-44fd-b6c2-cc9a6a455178","Type":"ContainerStarted","Data":"2ca813ded920368d0d5fdf4c79beb2d6db8429d617723880a37cfe7fbfef4d8d"}
Jul 28 22:09:39.634053 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:39.634016    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v" event={"ID":"b03a58fd-5c5b-44fd-b6c2-cc9a6a455178","Type":"ContainerStarted","Data":"ac7c04ff2779803c8c5c50532919b7856b0e1356e117ce97d606bb9be897cf8d"}
Jul 28 22:09:40.638842 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:40.638796    2574 generic.go:356] "Generic (PLEG): container finished" podID="b03a58fd-5c5b-44fd-b6c2-cc9a6a455178" containerID="ac7c04ff2779803c8c5c50532919b7856b0e1356e117ce97d606bb9be897cf8d" exitCode=0
Jul 28 22:09:40.639223 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:40.638885    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v" event={"ID":"b03a58fd-5c5b-44fd-b6c2-cc9a6a455178","Type":"ContainerDied","Data":"ac7c04ff2779803c8c5c50532919b7856b0e1356e117ce97d606bb9be897cf8d"}
Jul 28 22:09:41.643946 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:41.643904    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v" event={"ID":"b03a58fd-5c5b-44fd-b6c2-cc9a6a455178","Type":"ContainerStarted","Data":"b085674cdf3f3287dae06ec1efdf53b91f16e1ad01210ec30d94129e23a9da5a"}
Jul 28 22:09:41.644450 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:41.644019    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v"
Jul 28 22:09:41.667433 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:41.667378    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v" podStartSLOduration=2.397816517 podStartE2EDuration="6.667358626s" totalImagesPullingTime="3.245968876s" totalInitContainerRuntime="1.023573233s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:09:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:09:41.666532118 +0000 UTC m=+310.650674738" watchObservedRunningTime="2026-07-28 22:09:41.667358626 +0000 UTC m=+310.651501246"
Jul 28 22:09:52.650249 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:52.650204    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d45996775-fzq6v"
Jul 28 22:09:55.383287 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:55.383249    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xpb8v"]
Jul 28 22:09:55.386100 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:55.386083    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-xpb8v"
Jul 28 22:09:55.388038 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:55.388017    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 28 22:09:55.388343 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:55.388330    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-lwjhl\""
Jul 28 22:09:55.402929 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:55.402902    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xpb8v"]
Jul 28 22:09:55.422997 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:55.422962    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlhtp\" (UniqueName: \"kubernetes.io/projected/e0162e33-5b05-4e2e-a9a3-c58e95c401dd-kube-api-access-mlhtp\") pod \"odh-model-controller-7998c67dc4-xpb8v\" (UID: \"e0162e33-5b05-4e2e-a9a3-c58e95c401dd\") " pod="opendatahub/odh-model-controller-7998c67dc4-xpb8v"
Jul 28 22:09:55.423149 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:55.423002    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0162e33-5b05-4e2e-a9a3-c58e95c401dd-cert\") pod \"odh-model-controller-7998c67dc4-xpb8v\" (UID: \"e0162e33-5b05-4e2e-a9a3-c58e95c401dd\") " pod="opendatahub/odh-model-controller-7998c67dc4-xpb8v"
Jul 28 22:09:55.524291 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:55.524258    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlhtp\" (UniqueName: \"kubernetes.io/projected/e0162e33-5b05-4e2e-a9a3-c58e95c401dd-kube-api-access-mlhtp\") pod \"odh-model-controller-7998c67dc4-xpb8v\" (UID: \"e0162e33-5b05-4e2e-a9a3-c58e95c401dd\") " pod="opendatahub/odh-model-controller-7998c67dc4-xpb8v"
Jul 28 22:09:55.524468 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:55.524303    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0162e33-5b05-4e2e-a9a3-c58e95c401dd-cert\") pod \"odh-model-controller-7998c67dc4-xpb8v\" (UID: \"e0162e33-5b05-4e2e-a9a3-c58e95c401dd\") " pod="opendatahub/odh-model-controller-7998c67dc4-xpb8v"
Jul 28 22:09:55.524525 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:09:55.524471    2574 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 28 22:09:55.524561 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:09:55.524547    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0162e33-5b05-4e2e-a9a3-c58e95c401dd-cert podName:e0162e33-5b05-4e2e-a9a3-c58e95c401dd nodeName:}" failed. No retries permitted until 2026-07-28 22:09:56.024528884 +0000 UTC m=+325.008671487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e0162e33-5b05-4e2e-a9a3-c58e95c401dd-cert") pod "odh-model-controller-7998c67dc4-xpb8v" (UID: "e0162e33-5b05-4e2e-a9a3-c58e95c401dd") : secret "odh-model-controller-webhook-cert" not found
Jul 28 22:09:55.535148 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:55.535117    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlhtp\" (UniqueName: \"kubernetes.io/projected/e0162e33-5b05-4e2e-a9a3-c58e95c401dd-kube-api-access-mlhtp\") pod \"odh-model-controller-7998c67dc4-xpb8v\" (UID: \"e0162e33-5b05-4e2e-a9a3-c58e95c401dd\") " pod="opendatahub/odh-model-controller-7998c67dc4-xpb8v"
Jul 28 22:09:56.027486 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.027441    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0162e33-5b05-4e2e-a9a3-c58e95c401dd-cert\") pod \"odh-model-controller-7998c67dc4-xpb8v\" (UID: \"e0162e33-5b05-4e2e-a9a3-c58e95c401dd\") " pod="opendatahub/odh-model-controller-7998c67dc4-xpb8v"
Jul 28 22:09:56.029855 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.029821    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0162e33-5b05-4e2e-a9a3-c58e95c401dd-cert\") pod \"odh-model-controller-7998c67dc4-xpb8v\" (UID: \"e0162e33-5b05-4e2e-a9a3-c58e95c401dd\") " pod="opendatahub/odh-model-controller-7998c67dc4-xpb8v"
Jul 28 22:09:56.073015 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.072979    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5cd5d84f76-9pg4h"]
Jul 28 22:09:56.076222 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.076204    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-9pg4h"
Jul 28 22:09:56.078196 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.078170    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 22:09:56.078332 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.078241    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-bvl6c\""
Jul 28 22:09:56.078332 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.078243    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 22:09:56.078714 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.078697    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 22:09:56.078804 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.078699    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 22:09:56.087559 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.087538    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5cd5d84f76-9pg4h"]
Jul 28 22:09:56.128640 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.128605    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95ttt\" (UniqueName: \"kubernetes.io/projected/cc63bade-aba2-4894-97a9-d4fc058fad23-kube-api-access-95ttt\") pod \"kube-auth-proxy-5cd5d84f76-9pg4h\" (UID: \"cc63bade-aba2-4894-97a9-d4fc058fad23\") " pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-9pg4h"
Jul 28 22:09:56.128811 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.128668    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc63bade-aba2-4894-97a9-d4fc058fad23-tmp\") pod \"kube-auth-proxy-5cd5d84f76-9pg4h\" (UID: \"cc63bade-aba2-4894-97a9-d4fc058fad23\") " pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-9pg4h"
Jul 28 22:09:56.128811 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.128745    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc63bade-aba2-4894-97a9-d4fc058fad23-tls-certs\") pod \"kube-auth-proxy-5cd5d84f76-9pg4h\" (UID: \"cc63bade-aba2-4894-97a9-d4fc058fad23\") " pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-9pg4h"
Jul 28 22:09:56.201777 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.201737    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5nmvv"]
Jul 28 22:09:56.205039 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.205015    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-5nmvv"
Jul 28 22:09:56.206958 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.206934    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 28 22:09:56.207064 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.206991    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-2g8dz\""
Jul 28 22:09:56.214532 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.214506    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5nmvv"]
Jul 28 22:09:56.230121 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.230088    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95ttt\" (UniqueName: \"kubernetes.io/projected/cc63bade-aba2-4894-97a9-d4fc058fad23-kube-api-access-95ttt\") pod \"kube-auth-proxy-5cd5d84f76-9pg4h\" (UID: \"cc63bade-aba2-4894-97a9-d4fc058fad23\") " pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-9pg4h"
Jul 28 22:09:56.230276 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.230126    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpkhv\" (UniqueName: \"kubernetes.io/projected/f8b44918-780b-4433-8136-5bf279e14dd5-kube-api-access-dpkhv\") pod \"model-serving-api-54d89bd5fc-5nmvv\" (UID: \"f8b44918-780b-4433-8136-5bf279e14dd5\") " pod="opendatahub/model-serving-api-54d89bd5fc-5nmvv"
Jul 28 22:09:56.230276 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.230147    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8b44918-780b-4433-8136-5bf279e14dd5-tls-certs\") pod \"model-serving-api-54d89bd5fc-5nmvv\" (UID: \"f8b44918-780b-4433-8136-5bf279e14dd5\") " pod="opendatahub/model-serving-api-54d89bd5fc-5nmvv"
Jul 28 22:09:56.230358 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.230273    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc63bade-aba2-4894-97a9-d4fc058fad23-tmp\") pod \"kube-auth-proxy-5cd5d84f76-9pg4h\" (UID: \"cc63bade-aba2-4894-97a9-d4fc058fad23\") " pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-9pg4h"
Jul 28 22:09:56.230358 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.230315    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc63bade-aba2-4894-97a9-d4fc058fad23-tls-certs\") pod \"kube-auth-proxy-5cd5d84f76-9pg4h\" (UID: \"cc63bade-aba2-4894-97a9-d4fc058fad23\") " pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-9pg4h"
Jul 28 22:09:56.230466 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:09:56.230451    2574 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 28 22:09:56.230526 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:09:56.230516    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc63bade-aba2-4894-97a9-d4fc058fad23-tls-certs podName:cc63bade-aba2-4894-97a9-d4fc058fad23 nodeName:}" failed. No retries permitted until 2026-07-28 22:09:56.730500346 +0000 UTC m=+325.714642944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/cc63bade-aba2-4894-97a9-d4fc058fad23-tls-certs") pod "kube-auth-proxy-5cd5d84f76-9pg4h" (UID: "cc63bade-aba2-4894-97a9-d4fc058fad23") : secret "kube-auth-proxy-tls" not found
Jul 28 22:09:56.232478 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.232457    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc63bade-aba2-4894-97a9-d4fc058fad23-tmp\") pod \"kube-auth-proxy-5cd5d84f76-9pg4h\" (UID: \"cc63bade-aba2-4894-97a9-d4fc058fad23\") " pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-9pg4h"
Jul 28 22:09:56.241113 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.241093    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95ttt\" (UniqueName: \"kubernetes.io/projected/cc63bade-aba2-4894-97a9-d4fc058fad23-kube-api-access-95ttt\") pod \"kube-auth-proxy-5cd5d84f76-9pg4h\" (UID: \"cc63bade-aba2-4894-97a9-d4fc058fad23\") " pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-9pg4h"
Jul 28 22:09:56.295457 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.295363    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-xpb8v"
Jul 28 22:09:56.331717 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.331678    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpkhv\" (UniqueName: \"kubernetes.io/projected/f8b44918-780b-4433-8136-5bf279e14dd5-kube-api-access-dpkhv\") pod \"model-serving-api-54d89bd5fc-5nmvv\" (UID: \"f8b44918-780b-4433-8136-5bf279e14dd5\") " pod="opendatahub/model-serving-api-54d89bd5fc-5nmvv"
Jul 28 22:09:56.331717 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.331728    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8b44918-780b-4433-8136-5bf279e14dd5-tls-certs\") pod \"model-serving-api-54d89bd5fc-5nmvv\" (UID: \"f8b44918-780b-4433-8136-5bf279e14dd5\") " pod="opendatahub/model-serving-api-54d89bd5fc-5nmvv"
Jul 28 22:09:56.331978 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:09:56.331926    2574 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 28 22:09:56.332035 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:09:56.332017    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8b44918-780b-4433-8136-5bf279e14dd5-tls-certs podName:f8b44918-780b-4433-8136-5bf279e14dd5 nodeName:}" failed. No retries permitted until 2026-07-28 22:09:56.831992568 +0000 UTC m=+325.816135167 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/f8b44918-780b-4433-8136-5bf279e14dd5-tls-certs") pod "model-serving-api-54d89bd5fc-5nmvv" (UID: "f8b44918-780b-4433-8136-5bf279e14dd5") : secret "model-serving-api-tls" not found
Jul 28 22:09:56.382426 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.382395    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpkhv\" (UniqueName: \"kubernetes.io/projected/f8b44918-780b-4433-8136-5bf279e14dd5-kube-api-access-dpkhv\") pod \"model-serving-api-54d89bd5fc-5nmvv\" (UID: \"f8b44918-780b-4433-8136-5bf279e14dd5\") " pod="opendatahub/model-serving-api-54d89bd5fc-5nmvv"
Jul 28 22:09:56.420198 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.420166    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xpb8v"]
Jul 28 22:09:56.423517 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:09:56.423484    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0162e33_5b05_4e2e_a9a3_c58e95c401dd.slice/crio-2e44432197488c6b9d79d0dd78a99ec20ffc7b3380ad51b442c81560d4d555ae WatchSource:0}: Error finding container 2e44432197488c6b9d79d0dd78a99ec20ffc7b3380ad51b442c81560d4d555ae: Status 404 returned error can't find the container with id 2e44432197488c6b9d79d0dd78a99ec20ffc7b3380ad51b442c81560d4d555ae
Jul 28 22:09:56.698864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.698833    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-xpb8v" event={"ID":"e0162e33-5b05-4e2e-a9a3-c58e95c401dd","Type":"ContainerStarted","Data":"2e44432197488c6b9d79d0dd78a99ec20ffc7b3380ad51b442c81560d4d555ae"}
Jul 28 22:09:56.735168 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.735131    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc63bade-aba2-4894-97a9-d4fc058fad23-tls-certs\") pod \"kube-auth-proxy-5cd5d84f76-9pg4h\" (UID: \"cc63bade-aba2-4894-97a9-d4fc058fad23\") " pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-9pg4h"
Jul 28 22:09:56.737673 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.737645    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc63bade-aba2-4894-97a9-d4fc058fad23-tls-certs\") pod \"kube-auth-proxy-5cd5d84f76-9pg4h\" (UID: \"cc63bade-aba2-4894-97a9-d4fc058fad23\") " pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-9pg4h"
Jul 28 22:09:56.836131 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.836096    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8b44918-780b-4433-8136-5bf279e14dd5-tls-certs\") pod \"model-serving-api-54d89bd5fc-5nmvv\" (UID: \"f8b44918-780b-4433-8136-5bf279e14dd5\") " pod="opendatahub/model-serving-api-54d89bd5fc-5nmvv"
Jul 28 22:09:56.838510 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.838490    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8b44918-780b-4433-8136-5bf279e14dd5-tls-certs\") pod \"model-serving-api-54d89bd5fc-5nmvv\" (UID: \"f8b44918-780b-4433-8136-5bf279e14dd5\") " pod="opendatahub/model-serving-api-54d89bd5fc-5nmvv"
Jul 28 22:09:56.986376 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:56.986287    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-9pg4h"
Jul 28 22:09:57.115731 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.115697    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-5nmvv"
Jul 28 22:09:57.141588 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.141476    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5cd5d84f76-9pg4h"]
Jul 28 22:09:57.144730 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:09:57.144691    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc63bade_aba2_4894_97a9_d4fc058fad23.slice/crio-a72aed81f4f97fcb17f1126438a32219669c6204a67153b82950d6ca56c58dc0 WatchSource:0}: Error finding container a72aed81f4f97fcb17f1126438a32219669c6204a67153b82950d6ca56c58dc0: Status 404 returned error can't find the container with id a72aed81f4f97fcb17f1126438a32219669c6204a67153b82950d6ca56c58dc0
Jul 28 22:09:57.283881 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.283711    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5nmvv"]
Jul 28 22:09:57.288645 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:09:57.288617    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8b44918_780b_4433_8136_5bf279e14dd5.slice/crio-15bd968526df69ca1f36a0ef920aa825b838d0623f6455fb4cb6fb79d0c46bf0 WatchSource:0}: Error finding container 15bd968526df69ca1f36a0ef920aa825b838d0623f6455fb4cb6fb79d0c46bf0: Status 404 returned error can't find the container with id 15bd968526df69ca1f36a0ef920aa825b838d0623f6455fb4cb6fb79d0c46bf0
Jul 28 22:09:57.295876 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.295846    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"]
Jul 28 22:09:57.300584 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.300515    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.302803 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.302783    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 28 22:09:57.302905 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.302816    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 28 22:09:57.303096 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.303047    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 22:09:57.303205 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.303108    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 28 22:09:57.303205 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.303056    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-rfnr8\""
Jul 28 22:09:57.310167 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.310072    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"]
Jul 28 22:09:57.341292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.341258    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a359d264-251a-4201-a33c-0e035a3901ad-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-k7bfc\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.341478 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.341309    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a359d264-251a-4201-a33c-0e035a3901ad-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-k7bfc\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.341478 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.341340    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a359d264-251a-4201-a33c-0e035a3901ad-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-k7bfc\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.341478 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.341377    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a359d264-251a-4201-a33c-0e035a3901ad-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-k7bfc\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.341478 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.341424    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk54r\" (UniqueName: \"kubernetes.io/projected/a359d264-251a-4201-a33c-0e035a3901ad-kube-api-access-jk54r\") pod \"istiod-openshift-gateway-7b7995d5cd-k7bfc\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.341478 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.341452    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a359d264-251a-4201-a33c-0e035a3901ad-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-k7bfc\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.341703 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.341484    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a359d264-251a-4201-a33c-0e035a3901ad-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-k7bfc\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.442707 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.442672    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a359d264-251a-4201-a33c-0e035a3901ad-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-k7bfc\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.443174 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.442722    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a359d264-251a-4201-a33c-0e035a3901ad-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-k7bfc\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.443174 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.442755    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a359d264-251a-4201-a33c-0e035a3901ad-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-k7bfc\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.443174 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.442797    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a359d264-251a-4201-a33c-0e035a3901ad-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-k7bfc\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.443174 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.442843    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk54r\" (UniqueName: \"kubernetes.io/projected/a359d264-251a-4201-a33c-0e035a3901ad-kube-api-access-jk54r\") pod \"istiod-openshift-gateway-7b7995d5cd-k7bfc\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.443174 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.442875    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a359d264-251a-4201-a33c-0e035a3901ad-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-k7bfc\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.443174 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.442907    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a359d264-251a-4201-a33c-0e035a3901ad-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-k7bfc\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.444582 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.444542    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a359d264-251a-4201-a33c-0e035a3901ad-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-k7bfc\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.446226 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.446190    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a359d264-251a-4201-a33c-0e035a3901ad-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-k7bfc\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.446598 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.446579    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a359d264-251a-4201-a33c-0e035a3901ad-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-k7bfc\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.450664 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.450641    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a359d264-251a-4201-a33c-0e035a3901ad-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-k7bfc\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.453206 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.453143    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a359d264-251a-4201-a33c-0e035a3901ad-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-k7bfc\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.453401 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.453378    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a359d264-251a-4201-a33c-0e035a3901ad-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-k7bfc\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.454349 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.454327    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk54r\" (UniqueName: \"kubernetes.io/projected/a359d264-251a-4201-a33c-0e035a3901ad-kube-api-access-jk54r\") pod \"istiod-openshift-gateway-7b7995d5cd-k7bfc\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.613832 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.613751    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:09:57.704325 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.704285    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-5nmvv" event={"ID":"f8b44918-780b-4433-8136-5bf279e14dd5","Type":"ContainerStarted","Data":"15bd968526df69ca1f36a0ef920aa825b838d0623f6455fb4cb6fb79d0c46bf0"}
Jul 28 22:09:57.705604 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.705575    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-9pg4h" event={"ID":"cc63bade-aba2-4894-97a9-d4fc058fad23","Type":"ContainerStarted","Data":"a72aed81f4f97fcb17f1126438a32219669c6204a67153b82950d6ca56c58dc0"}
Jul 28 22:09:57.769119 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:57.769088    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"]
Jul 28 22:09:57.771531 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:09:57.771493    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda359d264_251a_4201_a33c_0e035a3901ad.slice/crio-52481cad08f5897b1e0e100f370bf2df1fc8b84d247a594e55cda23fc7cfd38e WatchSource:0}: Error finding container 52481cad08f5897b1e0e100f370bf2df1fc8b84d247a594e55cda23fc7cfd38e: Status 404 returned error can't find the container with id 52481cad08f5897b1e0e100f370bf2df1fc8b84d247a594e55cda23fc7cfd38e
Jul 28 22:09:58.712157 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:09:58.712118    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc" event={"ID":"a359d264-251a-4201-a33c-0e035a3901ad","Type":"ContainerStarted","Data":"52481cad08f5897b1e0e100f370bf2df1fc8b84d247a594e55cda23fc7cfd38e"}
Jul 28 22:10:00.723402 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:00.723368    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-xpb8v" event={"ID":"e0162e33-5b05-4e2e-a9a3-c58e95c401dd","Type":"ContainerStarted","Data":"8e554b9c1384b1cf5bd70e1f9a3555217335383b35cbdd3e1cfb2584306beaed"}
Jul 28 22:10:00.723864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:00.723469    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-xpb8v"
Jul 28 22:10:00.725288 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:00.725260    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-5nmvv" event={"ID":"f8b44918-780b-4433-8136-5bf279e14dd5","Type":"ContainerStarted","Data":"32b77f711cf374ca2e8c4ce3255d58c55eeeab27ed58b51702029c06ddd3313e"}
Jul 28 22:10:00.725405 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:00.725322    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-5nmvv"
Jul 28 22:10:00.726961 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:00.726934    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-9pg4h" event={"ID":"cc63bade-aba2-4894-97a9-d4fc058fad23","Type":"ContainerStarted","Data":"92172b5b44f1ec6a7a3c1e4eee5dc60ccc0c71f52a62896057a7321f0fb2d858"}
Jul 28 22:10:00.743435 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:00.742613    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-xpb8v" podStartSLOduration=1.640507465 podStartE2EDuration="5.742592123s" totalImagesPullingTime="4.102084658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:09:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:00.74078245 +0000 UTC m=+329.724925071" watchObservedRunningTime="2026-07-28 22:10:00.742592123 +0000 UTC m=+329.726734745"
Jul 28 22:10:00.756570 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:00.756507    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-9pg4h" podStartSLOduration=1.370289007 podStartE2EDuration="4.756487276s" totalImagesPullingTime="3.386198269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:09:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:00.755977583 +0000 UTC m=+329.740120215" watchObservedRunningTime="2026-07-28 22:10:00.756487276 +0000 UTC m=+329.740629897"
Jul 28 22:10:00.772461 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:00.772331    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-5nmvv" podStartSLOduration=1.536549674 podStartE2EDuration="4.772310656s" totalImagesPullingTime="3.235760982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:09:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:00.771867296 +0000 UTC m=+329.756009917" watchObservedRunningTime="2026-07-28 22:10:00.772310656 +0000 UTC m=+329.756453293"
Jul 28 22:10:01.862815 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:01.862782    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:10:01.863068 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:01.862847    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:10:02.736569 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:02.736536    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc" event={"ID":"a359d264-251a-4201-a33c-0e035a3901ad","Type":"ContainerStarted","Data":"d91888b8e607c636500ce6661587450ff76004b1e5dcbadfeb358be1464b5b67"}
Jul 28 22:10:02.736765 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:02.736744    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:10:02.738457 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:02.738429    2574 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-k7bfc container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 22:10:02.738573 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:02.738473    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc" podUID="a359d264-251a-4201-a33c-0e035a3901ad" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 22:10:02.761272 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:02.761203    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc" podStartSLOduration=1.6724477169999998 podStartE2EDuration="5.761183152s" totalImagesPullingTime="4.088735435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:09:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:02.76022748 +0000 UTC m=+331.744370096" watchObservedRunningTime="2026-07-28 22:10:02.761183152 +0000 UTC m=+331.745325773"
Jul 28 22:10:03.311926 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.311888    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"]
Jul 28 22:10:03.315098 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.315074    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.316964 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.316947    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-jw8fx\""
Jul 28 22:10:03.328020 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.327997    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"]
Jul 28 22:10:03.395550 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.395513    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.395712 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.395555    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzszf\" (UniqueName: \"kubernetes.io/projected/07d10cbe-a030-4e27-9db1-5f6296788ca1-kube-api-access-lzszf\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.395712 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.395581    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.395712 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.395606    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.395712 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.395629    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.395712 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.395645    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.395712 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.395659    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.395899 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.395781    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.395899 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.395809    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07d10cbe-a030-4e27-9db1-5f6296788ca1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.417109 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.417073    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"]
Jul 28 22:10:03.420303 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.420285    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.451997 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.451965    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"]
Jul 28 22:10:03.497216 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.497182    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1262efc1-7845-4349-8425-0609954af5c6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.497393 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.497225    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1262efc1-7845-4349-8425-0609954af5c6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.497393 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.497305    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1262efc1-7845-4349-8425-0609954af5c6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.497393 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.497349    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1262efc1-7845-4349-8425-0609954af5c6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.497589 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.497403    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1262efc1-7845-4349-8425-0609954af5c6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.497589 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.497437    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.497589 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.497461    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07d10cbe-a030-4e27-9db1-5f6296788ca1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.497589 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.497517    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1262efc1-7845-4349-8425-0609954af5c6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.497589 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.497578    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1262efc1-7845-4349-8425-0609954af5c6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.497813 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.497636    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.497813 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.497695    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1262efc1-7845-4349-8425-0609954af5c6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.497813 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.497739    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzszf\" (UniqueName: \"kubernetes.io/projected/07d10cbe-a030-4e27-9db1-5f6296788ca1-kube-api-access-lzszf\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.497813 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.497774    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.497813 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.497802    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.498054 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.497827    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.498054 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.497852    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.498054 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.497876    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.498054 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.497899    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngp9d\" (UniqueName: \"kubernetes.io/projected/1262efc1-7845-4349-8425-0609954af5c6-kube-api-access-ngp9d\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.498054 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.497961    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.498313 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.498262    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07d10cbe-a030-4e27-9db1-5f6296788ca1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.498313 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.498288    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.498414 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.498332    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.498414 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.498398    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.499958 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.499932    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.500417 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.500400    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.514642 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.514613    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.521360 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.521334    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzszf\" (UniqueName: \"kubernetes.io/projected/07d10cbe-a030-4e27-9db1-5f6296788ca1-kube-api-access-lzszf\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-78g42\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.599129 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.599043    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngp9d\" (UniqueName: \"kubernetes.io/projected/1262efc1-7845-4349-8425-0609954af5c6-kube-api-access-ngp9d\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.599129 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.599087    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1262efc1-7845-4349-8425-0609954af5c6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.599129 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.599118    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1262efc1-7845-4349-8425-0609954af5c6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.599423 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.599144    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1262efc1-7845-4349-8425-0609954af5c6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.599423 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.599174    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1262efc1-7845-4349-8425-0609954af5c6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.599423 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.599313    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1262efc1-7845-4349-8425-0609954af5c6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.599423 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.599377    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1262efc1-7845-4349-8425-0609954af5c6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.599589 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.599444    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1262efc1-7845-4349-8425-0609954af5c6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.599637 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.599605    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1262efc1-7845-4349-8425-0609954af5c6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.599737 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.599705    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1262efc1-7845-4349-8425-0609954af5c6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.599865 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.599793    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1262efc1-7845-4349-8425-0609954af5c6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.599865 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.599834    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1262efc1-7845-4349-8425-0609954af5c6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.600180 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.600156    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1262efc1-7845-4349-8425-0609954af5c6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.600289 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.600185    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1262efc1-7845-4349-8425-0609954af5c6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.601854 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.601833    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1262efc1-7845-4349-8425-0609954af5c6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.602059 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.602039    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1262efc1-7845-4349-8425-0609954af5c6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.616884 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.616861    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngp9d\" (UniqueName: \"kubernetes.io/projected/1262efc1-7845-4349-8425-0609954af5c6-kube-api-access-ngp9d\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.619504 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.619484    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1262efc1-7845-4349-8425-0609954af5c6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799crd6zz\" (UID: \"1262efc1-7845-4349-8425-0609954af5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.626479 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.626459    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:03.730426 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.730396    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:03.743010 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.742983    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:10:03.766970 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.766859    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"]
Jul 28 22:10:03.769501 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:10:03.769475    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07d10cbe_a030_4e27_9db1_5f6296788ca1.slice/crio-9eb13ad8a0807a8a6e7392ab608ff6369dfc47ea50b0b24341119302f821bc9f WatchSource:0}: Error finding container 9eb13ad8a0807a8a6e7392ab608ff6369dfc47ea50b0b24341119302f821bc9f: Status 404 returned error can't find the container with id 9eb13ad8a0807a8a6e7392ab608ff6369dfc47ea50b0b24341119302f821bc9f
Jul 28 22:10:03.875743 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:03.875548    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"]
Jul 28 22:10:03.880070 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:10:03.880005    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1262efc1_7845_4349_8425_0609954af5c6.slice/crio-cae3d9cb030eae1e752a9ddbb2dd128c91430f4d30197d723f237984d1dcd234 WatchSource:0}: Error finding container cae3d9cb030eae1e752a9ddbb2dd128c91430f4d30197d723f237984d1dcd234: Status 404 returned error can't find the container with id cae3d9cb030eae1e752a9ddbb2dd128c91430f4d30197d723f237984d1dcd234
Jul 28 22:10:04.746842 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:04.746802    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz" event={"ID":"1262efc1-7845-4349-8425-0609954af5c6","Type":"ContainerStarted","Data":"cae3d9cb030eae1e752a9ddbb2dd128c91430f4d30197d723f237984d1dcd234"}
Jul 28 22:10:04.747994 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:04.747970    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42" event={"ID":"07d10cbe-a030-4e27-9db1-5f6296788ca1","Type":"ContainerStarted","Data":"9eb13ad8a0807a8a6e7392ab608ff6369dfc47ea50b0b24341119302f821bc9f"}
Jul 28 22:10:05.779492 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:05.779394    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-cp5wh"]
Jul 28 22:10:05.783094 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:05.783005    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cp5wh"
Jul 28 22:10:05.790623 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:05.790376    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 28 22:10:05.790623 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:05.790401    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-5c628\""
Jul 28 22:10:05.791755 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:05.791606    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jvrt4"]
Jul 28 22:10:05.796389 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:05.796332    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvrt4"
Jul 28 22:10:05.800060 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:05.800037    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 28 22:10:05.800060 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:05.800057    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-7fd2q\""
Jul 28 22:10:05.805096 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:05.805029    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-cp5wh"]
Jul 28 22:10:05.815610 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:05.815583    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jvrt4"]
Jul 28 22:10:05.925481 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:05.925439    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/21574045-310f-4d71-9427-477488ae7bbb-cert\") pod \"kserve-controller-manager-d6f4b65b8-cp5wh\" (UID: \"21574045-310f-4d71-9427-477488ae7bbb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cp5wh"
Jul 28 22:10:05.925729 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:05.925527    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbl99\" (UniqueName: \"kubernetes.io/projected/846482be-f263-409b-a5f2-a290e50d2cb8-kube-api-access-mbl99\") pod \"llmisvc-controller-manager-777c6bd7d5-jvrt4\" (UID: \"846482be-f263-409b-a5f2-a290e50d2cb8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvrt4"
Jul 28 22:10:05.925729 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:05.925561    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/846482be-f263-409b-a5f2-a290e50d2cb8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jvrt4\" (UID: \"846482be-f263-409b-a5f2-a290e50d2cb8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvrt4"
Jul 28 22:10:05.925729 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:05.925694    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4gtw\" (UniqueName: \"kubernetes.io/projected/21574045-310f-4d71-9427-477488ae7bbb-kube-api-access-k4gtw\") pod \"kserve-controller-manager-d6f4b65b8-cp5wh\" (UID: \"21574045-310f-4d71-9427-477488ae7bbb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cp5wh"
Jul 28 22:10:06.026777 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:06.026588    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/21574045-310f-4d71-9427-477488ae7bbb-cert\") pod \"kserve-controller-manager-d6f4b65b8-cp5wh\" (UID: \"21574045-310f-4d71-9427-477488ae7bbb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cp5wh"
Jul 28 22:10:06.026777 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:06.026690    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbl99\" (UniqueName: \"kubernetes.io/projected/846482be-f263-409b-a5f2-a290e50d2cb8-kube-api-access-mbl99\") pod \"llmisvc-controller-manager-777c6bd7d5-jvrt4\" (UID: \"846482be-f263-409b-a5f2-a290e50d2cb8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvrt4"
Jul 28 22:10:06.026777 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:06.026723    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/846482be-f263-409b-a5f2-a290e50d2cb8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jvrt4\" (UID: \"846482be-f263-409b-a5f2-a290e50d2cb8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvrt4"
Jul 28 22:10:06.027061 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:06.026796    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4gtw\" (UniqueName: \"kubernetes.io/projected/21574045-310f-4d71-9427-477488ae7bbb-kube-api-access-k4gtw\") pod \"kserve-controller-manager-d6f4b65b8-cp5wh\" (UID: \"21574045-310f-4d71-9427-477488ae7bbb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cp5wh"
Jul 28 22:10:06.027125 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:10:06.027074    2574 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 28 22:10:06.027171 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:10:06.027145    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/846482be-f263-409b-a5f2-a290e50d2cb8-cert podName:846482be-f263-409b-a5f2-a290e50d2cb8 nodeName:}" failed. No retries permitted until 2026-07-28 22:10:06.527124553 +0000 UTC m=+335.511267151 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/846482be-f263-409b-a5f2-a290e50d2cb8-cert") pod "llmisvc-controller-manager-777c6bd7d5-jvrt4" (UID: "846482be-f263-409b-a5f2-a290e50d2cb8") : secret "llmisvc-webhook-server-cert" not found
Jul 28 22:10:06.030411 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:06.030333    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/21574045-310f-4d71-9427-477488ae7bbb-cert\") pod \"kserve-controller-manager-d6f4b65b8-cp5wh\" (UID: \"21574045-310f-4d71-9427-477488ae7bbb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cp5wh"
Jul 28 22:10:06.037284 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:06.037213    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4gtw\" (UniqueName: \"kubernetes.io/projected/21574045-310f-4d71-9427-477488ae7bbb-kube-api-access-k4gtw\") pod \"kserve-controller-manager-d6f4b65b8-cp5wh\" (UID: \"21574045-310f-4d71-9427-477488ae7bbb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cp5wh"
Jul 28 22:10:06.037525 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:06.037506    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbl99\" (UniqueName: \"kubernetes.io/projected/846482be-f263-409b-a5f2-a290e50d2cb8-kube-api-access-mbl99\") pod \"llmisvc-controller-manager-777c6bd7d5-jvrt4\" (UID: \"846482be-f263-409b-a5f2-a290e50d2cb8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvrt4"
Jul 28 22:10:06.100395 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:06.100352    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cp5wh"
Jul 28 22:10:06.533040 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:06.532964    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/846482be-f263-409b-a5f2-a290e50d2cb8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jvrt4\" (UID: \"846482be-f263-409b-a5f2-a290e50d2cb8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvrt4"
Jul 28 22:10:06.535950 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:06.535920    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/846482be-f263-409b-a5f2-a290e50d2cb8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jvrt4\" (UID: \"846482be-f263-409b-a5f2-a290e50d2cb8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvrt4"
Jul 28 22:10:06.711377 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:06.711338    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvrt4"
Jul 28 22:10:07.224110 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:07.224068    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:10:07.224542 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:07.224168    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:10:07.224542 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:07.224210    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:10:07.230763 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:07.230727    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:10:07.230883 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:07.230807    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:10:07.230944 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:07.230882    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:10:07.317690 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:07.317653    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jvrt4"]
Jul 28 22:10:07.323262 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:10:07.323210    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod846482be_f263_409b_a5f2_a290e50d2cb8.slice/crio-4a876753fd28bcb8f4473cf67ac7388615d2eb6fb9e779f757d5691f9d6236f6 WatchSource:0}: Error finding container 4a876753fd28bcb8f4473cf67ac7388615d2eb6fb9e779f757d5691f9d6236f6: Status 404 returned error can't find the container with id 4a876753fd28bcb8f4473cf67ac7388615d2eb6fb9e779f757d5691f9d6236f6
Jul 28 22:10:07.349038 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:07.348786    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-cp5wh"]
Jul 28 22:10:07.350988 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:10:07.350956    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21574045_310f_4d71_9427_477488ae7bbb.slice/crio-05f4c09e9853eaf40065297413ad24853deb78b8ced7fcfc3f7337986a04cf6b WatchSource:0}: Error finding container 05f4c09e9853eaf40065297413ad24853deb78b8ced7fcfc3f7337986a04cf6b: Status 404 returned error can't find the container with id 05f4c09e9853eaf40065297413ad24853deb78b8ced7fcfc3f7337986a04cf6b
Jul 28 22:10:07.671034 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:07.670999    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"]
Jul 28 22:10:07.671205 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:07.671172    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc" podUID="a359d264-251a-4201-a33c-0e035a3901ad" containerName="discovery" containerID="cri-o://d91888b8e607c636500ce6661587450ff76004b1e5dcbadfeb358be1464b5b67" gracePeriod=30
Jul 28 22:10:07.764930 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:07.764888    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42" event={"ID":"07d10cbe-a030-4e27-9db1-5f6296788ca1","Type":"ContainerStarted","Data":"3678255f3f9116b4dcc284057ca509288504e7f05654aa1f93e486b15d026371"}
Jul 28 22:10:07.766104 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:07.766077    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvrt4" event={"ID":"846482be-f263-409b-a5f2-a290e50d2cb8","Type":"ContainerStarted","Data":"4a876753fd28bcb8f4473cf67ac7388615d2eb6fb9e779f757d5691f9d6236f6"}
Jul 28 22:10:07.767542 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:07.767517    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz" event={"ID":"1262efc1-7845-4349-8425-0609954af5c6","Type":"ContainerStarted","Data":"cbf26836f947574c7cae21350e7f1ef2924135fbba2989ba840c544b07d64f08"}
Jul 28 22:10:07.768817 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:07.768795    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cp5wh" event={"ID":"21574045-310f-4d71-9427-477488ae7bbb","Type":"ContainerStarted","Data":"05f4c09e9853eaf40065297413ad24853deb78b8ced7fcfc3f7337986a04cf6b"}
Jul 28 22:10:07.788357 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:07.788310    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42" podStartSLOduration=1.32934661 podStartE2EDuration="4.788293446s" totalImagesPullingTime="3.458946836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:10:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:07.786364597 +0000 UTC m=+336.770507218" watchObservedRunningTime="2026-07-28 22:10:07.788293446 +0000 UTC m=+336.772436066"
Jul 28 22:10:07.808501 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:07.808447    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz" podStartSLOduration=1.466552337 podStartE2EDuration="4.808427875s" totalImagesPullingTime="3.341875538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:10:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:07.807110131 +0000 UTC m=+336.791252751" watchObservedRunningTime="2026-07-28 22:10:07.808427875 +0000 UTC m=+336.792570495"
Jul 28 22:10:07.911074 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:07.911050    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:10:08.046529 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.046476    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jk54r\" (UniqueName: \"kubernetes.io/projected/a359d264-251a-4201-a33c-0e035a3901ad-kube-api-access-jk54r\") pod \"a359d264-251a-4201-a33c-0e035a3901ad\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") "
Jul 28 22:10:08.046695 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.046541    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a359d264-251a-4201-a33c-0e035a3901ad-local-certs\") pod \"a359d264-251a-4201-a33c-0e035a3901ad\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") "
Jul 28 22:10:08.046695 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.046572    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a359d264-251a-4201-a33c-0e035a3901ad-cacerts\") pod \"a359d264-251a-4201-a33c-0e035a3901ad\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") "
Jul 28 22:10:08.046695 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.046609    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a359d264-251a-4201-a33c-0e035a3901ad-istio-csr-ca-configmap\") pod \"a359d264-251a-4201-a33c-0e035a3901ad\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") "
Jul 28 22:10:08.046695 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.046642    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a359d264-251a-4201-a33c-0e035a3901ad-istio-token\") pod \"a359d264-251a-4201-a33c-0e035a3901ad\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") "
Jul 28 22:10:08.046695 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.046665    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a359d264-251a-4201-a33c-0e035a3901ad-istio-kubeconfig\") pod \"a359d264-251a-4201-a33c-0e035a3901ad\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") "
Jul 28 22:10:08.047001 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.046725    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a359d264-251a-4201-a33c-0e035a3901ad-istio-csr-dns-cert\") pod \"a359d264-251a-4201-a33c-0e035a3901ad\" (UID: \"a359d264-251a-4201-a33c-0e035a3901ad\") "
Jul 28 22:10:08.047135 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.047042    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a359d264-251a-4201-a33c-0e035a3901ad-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "a359d264-251a-4201-a33c-0e035a3901ad" (UID: "a359d264-251a-4201-a33c-0e035a3901ad"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:10:08.050285 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.050204    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a359d264-251a-4201-a33c-0e035a3901ad-istio-token" (OuterVolumeSpecName: "istio-token") pod "a359d264-251a-4201-a33c-0e035a3901ad" (UID: "a359d264-251a-4201-a33c-0e035a3901ad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:10:08.050285 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.050258    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a359d264-251a-4201-a33c-0e035a3901ad-cacerts" (OuterVolumeSpecName: "cacerts") pod "a359d264-251a-4201-a33c-0e035a3901ad" (UID: "a359d264-251a-4201-a33c-0e035a3901ad"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:10:08.050471 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.050304    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a359d264-251a-4201-a33c-0e035a3901ad-kube-api-access-jk54r" (OuterVolumeSpecName: "kube-api-access-jk54r") pod "a359d264-251a-4201-a33c-0e035a3901ad" (UID: "a359d264-251a-4201-a33c-0e035a3901ad"). InnerVolumeSpecName "kube-api-access-jk54r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:10:08.050471 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.050343    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a359d264-251a-4201-a33c-0e035a3901ad-local-certs" (OuterVolumeSpecName: "local-certs") pod "a359d264-251a-4201-a33c-0e035a3901ad" (UID: "a359d264-251a-4201-a33c-0e035a3901ad"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:10:08.050675 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.050652    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a359d264-251a-4201-a33c-0e035a3901ad-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "a359d264-251a-4201-a33c-0e035a3901ad" (UID: "a359d264-251a-4201-a33c-0e035a3901ad"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:10:08.052404 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.052357    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a359d264-251a-4201-a33c-0e035a3901ad-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "a359d264-251a-4201-a33c-0e035a3901ad" (UID: "a359d264-251a-4201-a33c-0e035a3901ad"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:10:08.155451 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.155414    2574 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a359d264-251a-4201-a33c-0e035a3901ad-istio-csr-ca-configmap\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:10:08.155628 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.155462    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a359d264-251a-4201-a33c-0e035a3901ad-istio-token\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:10:08.155628 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.155479    2574 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a359d264-251a-4201-a33c-0e035a3901ad-istio-kubeconfig\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:10:08.155628 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.155494    2574 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a359d264-251a-4201-a33c-0e035a3901ad-istio-csr-dns-cert\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:10:08.155628 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.155512    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jk54r\" (UniqueName: \"kubernetes.io/projected/a359d264-251a-4201-a33c-0e035a3901ad-kube-api-access-jk54r\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:10:08.155628 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.155526    2574 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a359d264-251a-4201-a33c-0e035a3901ad-local-certs\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:10:08.155628 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.155540    2574 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a359d264-251a-4201-a33c-0e035a3901ad-cacerts\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:10:08.627787 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.627739    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:08.629296 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.629267    2574 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.19:15021/healthz/ready\": dial tcp 10.133.0.19:15021: connect: connection refused" start-of-body=
Jul 28 22:10:08.629414 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.629345    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42" podUID="07d10cbe-a030-4e27-9db1-5f6296788ca1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.19:15021/healthz/ready\": dial tcp 10.133.0.19:15021: connect: connection refused"
Jul 28 22:10:08.731189 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.731146    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:08.738365 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.738334    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:08.775645 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.775610    2574 generic.go:356] "Generic (PLEG): container finished" podID="a359d264-251a-4201-a33c-0e035a3901ad" containerID="d91888b8e607c636500ce6661587450ff76004b1e5dcbadfeb358be1464b5b67" exitCode=0
Jul 28 22:10:08.775813 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.775692    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"
Jul 28 22:10:08.775813 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.775760    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc" event={"ID":"a359d264-251a-4201-a33c-0e035a3901ad","Type":"ContainerDied","Data":"d91888b8e607c636500ce6661587450ff76004b1e5dcbadfeb358be1464b5b67"}
Jul 28 22:10:08.775953 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.775825    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc" event={"ID":"a359d264-251a-4201-a33c-0e035a3901ad","Type":"ContainerDied","Data":"52481cad08f5897b1e0e100f370bf2df1fc8b84d247a594e55cda23fc7cfd38e"}
Jul 28 22:10:08.775953 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.775854    2574 scope.go:117] "RemoveContainer" containerID="d91888b8e607c636500ce6661587450ff76004b1e5dcbadfeb358be1464b5b67"
Jul 28 22:10:08.776601 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.776576    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:08.777838 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.777787    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crd6zz"
Jul 28 22:10:08.786076 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.785901    2574 scope.go:117] "RemoveContainer" containerID="d91888b8e607c636500ce6661587450ff76004b1e5dcbadfeb358be1464b5b67"
Jul 28 22:10:08.786499 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:10:08.786428    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d91888b8e607c636500ce6661587450ff76004b1e5dcbadfeb358be1464b5b67\": container with ID starting with d91888b8e607c636500ce6661587450ff76004b1e5dcbadfeb358be1464b5b67 not found: ID does not exist" containerID="d91888b8e607c636500ce6661587450ff76004b1e5dcbadfeb358be1464b5b67"
Jul 28 22:10:08.786499 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.786470    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d91888b8e607c636500ce6661587450ff76004b1e5dcbadfeb358be1464b5b67"} err="failed to get container status \"d91888b8e607c636500ce6661587450ff76004b1e5dcbadfeb358be1464b5b67\": rpc error: code = NotFound desc = could not find container \"d91888b8e607c636500ce6661587450ff76004b1e5dcbadfeb358be1464b5b67\": container with ID starting with d91888b8e607c636500ce6661587450ff76004b1e5dcbadfeb358be1464b5b67 not found: ID does not exist"
Jul 28 22:10:08.813617 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.813217    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"]
Jul 28 22:10:08.815098 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.815072    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-k7bfc"]
Jul 28 22:10:08.834875 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:08.834779    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"]
Jul 28 22:10:09.495283 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:09.495251    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a359d264-251a-4201-a33c-0e035a3901ad" path="/var/lib/kubelet/pods/a359d264-251a-4201-a33c-0e035a3901ad/volumes"
Jul 28 22:10:09.626871 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:09.626844    2574 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.19:15021/healthz/ready\": dial tcp 10.133.0.19:15021: connect: connection refused" start-of-body=
Jul 28 22:10:09.626961 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:09.626916    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42" podUID="07d10cbe-a030-4e27-9db1-5f6296788ca1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.19:15021/healthz/ready\": dial tcp 10.133.0.19:15021: connect: connection refused"
Jul 28 22:10:09.784410 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:09.784323    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cp5wh" event={"ID":"21574045-310f-4d71-9427-477488ae7bbb","Type":"ContainerStarted","Data":"4d21d30395844b2929f1b0bce86a449cad0c191a459120336325c4abb6a98275"}
Jul 28 22:10:09.801559 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:09.801507    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cp5wh" podStartSLOduration=2.52754477 podStartE2EDuration="4.801488188s" totalImagesPullingTime="2.273943418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:10:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:09.799926343 +0000 UTC m=+338.784068985" watchObservedRunningTime="2026-07-28 22:10:09.801488188 +0000 UTC m=+338.785630810"
Jul 28 22:10:10.627352 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:10.627266    2574 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.19:15021/healthz/ready\": dial tcp 10.133.0.19:15021: connect: connection refused" start-of-body=
Jul 28 22:10:10.627352 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:10.627340    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42" podUID="07d10cbe-a030-4e27-9db1-5f6296788ca1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.19:15021/healthz/ready\": dial tcp 10.133.0.19:15021: connect: connection refused"
Jul 28 22:10:10.790300 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:10.790261    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvrt4" event={"ID":"846482be-f263-409b-a5f2-a290e50d2cb8","Type":"ContainerStarted","Data":"a6c43b9304981b85f110cf7bbfba619b4b4b164aae797fe2865c1d4e72d11e81"}
Jul 28 22:10:10.790757 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:10.790638    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cp5wh"
Jul 28 22:10:10.790757 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:10.790647    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42" podUID="07d10cbe-a030-4e27-9db1-5f6296788ca1" containerName="istio-proxy" containerID="cri-o://3678255f3f9116b4dcc284057ca509288504e7f05654aa1f93e486b15d026371" gracePeriod=30
Jul 28 22:10:10.790757 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:10.790678    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvrt4"
Jul 28 22:10:10.804787 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:10.804683    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvrt4" podStartSLOduration=2.824880882 podStartE2EDuration="5.804665359s" totalImagesPullingTime="2.979784477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:10:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:10.804275969 +0000 UTC m=+339.788418589" watchObservedRunningTime="2026-07-28 22:10:10.804665359 +0000 UTC m=+339.788807980"
Jul 28 22:10:11.735327 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:11.735294    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-5nmvv"
Jul 28 22:10:11.735647 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:11.735623    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-xpb8v"
Jul 28 22:10:16.028301 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.028277    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:16.129530 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.129437    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-data\") pod \"07d10cbe-a030-4e27-9db1-5f6296788ca1\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") "
Jul 28 22:10:16.129530 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.129487    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07d10cbe-a030-4e27-9db1-5f6296788ca1-istiod-ca-cert\") pod \"07d10cbe-a030-4e27-9db1-5f6296788ca1\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") "
Jul 28 22:10:16.129764 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.129537    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzszf\" (UniqueName: \"kubernetes.io/projected/07d10cbe-a030-4e27-9db1-5f6296788ca1-kube-api-access-lzszf\") pod \"07d10cbe-a030-4e27-9db1-5f6296788ca1\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") "
Jul 28 22:10:16.129764 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.129558    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-workload-certs\") pod \"07d10cbe-a030-4e27-9db1-5f6296788ca1\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") "
Jul 28 22:10:16.129764 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.129575    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-podinfo\") pod \"07d10cbe-a030-4e27-9db1-5f6296788ca1\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") "
Jul 28 22:10:16.129764 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.129603    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-credential-socket\") pod \"07d10cbe-a030-4e27-9db1-5f6296788ca1\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") "
Jul 28 22:10:16.129764 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.129715    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-data" (OuterVolumeSpecName: "istio-data") pod "07d10cbe-a030-4e27-9db1-5f6296788ca1" (UID: "07d10cbe-a030-4e27-9db1-5f6296788ca1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:10:16.129764 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.129761    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-token\") pod \"07d10cbe-a030-4e27-9db1-5f6296788ca1\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") "
Jul 28 22:10:16.130007 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.129833    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-envoy\") pod \"07d10cbe-a030-4e27-9db1-5f6296788ca1\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") "
Jul 28 22:10:16.130007 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.129873    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-workload-socket\") pod \"07d10cbe-a030-4e27-9db1-5f6296788ca1\" (UID: \"07d10cbe-a030-4e27-9db1-5f6296788ca1\") "
Jul 28 22:10:16.130007 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.129952    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "07d10cbe-a030-4e27-9db1-5f6296788ca1" (UID: "07d10cbe-a030-4e27-9db1-5f6296788ca1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:10:16.130007 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.129980    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07d10cbe-a030-4e27-9db1-5f6296788ca1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "07d10cbe-a030-4e27-9db1-5f6296788ca1" (UID: "07d10cbe-a030-4e27-9db1-5f6296788ca1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:10:16.130198 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.129985    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "07d10cbe-a030-4e27-9db1-5f6296788ca1" (UID: "07d10cbe-a030-4e27-9db1-5f6296788ca1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:10:16.130198 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.130189    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "07d10cbe-a030-4e27-9db1-5f6296788ca1" (UID: "07d10cbe-a030-4e27-9db1-5f6296788ca1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:10:16.130310 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.130292    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-workload-socket\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:10:16.130358 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.130315    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-data\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:10:16.130358 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.130329    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07d10cbe-a030-4e27-9db1-5f6296788ca1-istiod-ca-cert\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:10:16.130358 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.130341    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-workload-certs\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:10:16.130358 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.130354    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-credential-socket\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:10:16.131904 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.131880    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-token" (OuterVolumeSpecName: "istio-token") pod "07d10cbe-a030-4e27-9db1-5f6296788ca1" (UID: "07d10cbe-a030-4e27-9db1-5f6296788ca1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:10:16.132006 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.131903    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07d10cbe-a030-4e27-9db1-5f6296788ca1-kube-api-access-lzszf" (OuterVolumeSpecName: "kube-api-access-lzszf") pod "07d10cbe-a030-4e27-9db1-5f6296788ca1" (UID: "07d10cbe-a030-4e27-9db1-5f6296788ca1"). InnerVolumeSpecName "kube-api-access-lzszf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:10:16.132279 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.132260    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "07d10cbe-a030-4e27-9db1-5f6296788ca1" (UID: "07d10cbe-a030-4e27-9db1-5f6296788ca1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 22:10:16.132279 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.132265    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "07d10cbe-a030-4e27-9db1-5f6296788ca1" (UID: "07d10cbe-a030-4e27-9db1-5f6296788ca1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:10:16.231143 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.231087    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-envoy\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:10:16.231143 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.231137    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzszf\" (UniqueName: \"kubernetes.io/projected/07d10cbe-a030-4e27-9db1-5f6296788ca1-kube-api-access-lzszf\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:10:16.231143 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.231148    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-podinfo\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:10:16.231143 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.231156    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07d10cbe-a030-4e27-9db1-5f6296788ca1-istio-token\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:10:16.814759 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.814725    2574 generic.go:356] "Generic (PLEG): container finished" podID="07d10cbe-a030-4e27-9db1-5f6296788ca1" containerID="3678255f3f9116b4dcc284057ca509288504e7f05654aa1f93e486b15d026371" exitCode=0
Jul 28 22:10:16.814910 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.814776    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"
Jul 28 22:10:16.814910 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.814803    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42" event={"ID":"07d10cbe-a030-4e27-9db1-5f6296788ca1","Type":"ContainerDied","Data":"3678255f3f9116b4dcc284057ca509288504e7f05654aa1f93e486b15d026371"}
Jul 28 22:10:16.814910 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.814858    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42" event={"ID":"07d10cbe-a030-4e27-9db1-5f6296788ca1","Type":"ContainerDied","Data":"9eb13ad8a0807a8a6e7392ab608ff6369dfc47ea50b0b24341119302f821bc9f"}
Jul 28 22:10:16.814910 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.814881    2574 scope.go:117] "RemoveContainer" containerID="3678255f3f9116b4dcc284057ca509288504e7f05654aa1f93e486b15d026371"
Jul 28 22:10:16.823104 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.823086    2574 scope.go:117] "RemoveContainer" containerID="3678255f3f9116b4dcc284057ca509288504e7f05654aa1f93e486b15d026371"
Jul 28 22:10:16.823368 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:10:16.823349    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3678255f3f9116b4dcc284057ca509288504e7f05654aa1f93e486b15d026371\": container with ID starting with 3678255f3f9116b4dcc284057ca509288504e7f05654aa1f93e486b15d026371 not found: ID does not exist" containerID="3678255f3f9116b4dcc284057ca509288504e7f05654aa1f93e486b15d026371"
Jul 28 22:10:16.823416 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.823378    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3678255f3f9116b4dcc284057ca509288504e7f05654aa1f93e486b15d026371"} err="failed to get container status \"3678255f3f9116b4dcc284057ca509288504e7f05654aa1f93e486b15d026371\": rpc error: code = NotFound desc = could not find container \"3678255f3f9116b4dcc284057ca509288504e7f05654aa1f93e486b15d026371\": container with ID starting with 3678255f3f9116b4dcc284057ca509288504e7f05654aa1f93e486b15d026371 not found: ID does not exist"
Jul 28 22:10:16.834821 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.834795    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"]
Jul 28 22:10:16.838402 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:16.838381    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-78g42"]
Jul 28 22:10:17.492865 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:17.492833    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07d10cbe-a030-4e27-9db1-5f6296788ca1" path="/var/lib/kubelet/pods/07d10cbe-a030-4e27-9db1-5f6296788ca1/volumes"
Jul 28 22:10:41.800140 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:41.800104    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cp5wh"
Jul 28 22:10:42.801270 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:42.801218    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvrt4"
Jul 28 22:10:56.462319 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:56.462285    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-s7n8d"]
Jul 28 22:10:56.462696 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:56.462585    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07d10cbe-a030-4e27-9db1-5f6296788ca1" containerName="istio-proxy"
Jul 28 22:10:56.462696 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:56.462596    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="07d10cbe-a030-4e27-9db1-5f6296788ca1" containerName="istio-proxy"
Jul 28 22:10:56.462696 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:56.462609    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a359d264-251a-4201-a33c-0e035a3901ad" containerName="discovery"
Jul 28 22:10:56.462696 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:56.462614    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a359d264-251a-4201-a33c-0e035a3901ad" containerName="discovery"
Jul 28 22:10:56.462696 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:56.462659    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a359d264-251a-4201-a33c-0e035a3901ad" containerName="discovery"
Jul 28 22:10:56.462696 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:56.462667    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="07d10cbe-a030-4e27-9db1-5f6296788ca1" containerName="istio-proxy"
Jul 28 22:10:56.464386 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:56.464368    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-s7n8d"
Jul 28 22:10:56.466286 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:56.466263    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 22:10:56.466399 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:56.466356    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 22:10:56.466747 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:56.466732    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-g4wck\""
Jul 28 22:10:56.473634 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:56.473604    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-s7n8d"]
Jul 28 22:10:56.561689 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:56.561648    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4f5h\" (UniqueName: \"kubernetes.io/projected/ca09d9b7-61ee-4cb1-8ac9-23595634fcae-kube-api-access-p4f5h\") pod \"kuadrant-operator-catalog-s7n8d\" (UID: \"ca09d9b7-61ee-4cb1-8ac9-23595634fcae\") " pod="kuadrant-system/kuadrant-operator-catalog-s7n8d"
Jul 28 22:10:56.662467 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:56.662428    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4f5h\" (UniqueName: \"kubernetes.io/projected/ca09d9b7-61ee-4cb1-8ac9-23595634fcae-kube-api-access-p4f5h\") pod \"kuadrant-operator-catalog-s7n8d\" (UID: \"ca09d9b7-61ee-4cb1-8ac9-23595634fcae\") " pod="kuadrant-system/kuadrant-operator-catalog-s7n8d"
Jul 28 22:10:56.669640 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:56.669615    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4f5h\" (UniqueName: \"kubernetes.io/projected/ca09d9b7-61ee-4cb1-8ac9-23595634fcae-kube-api-access-p4f5h\") pod \"kuadrant-operator-catalog-s7n8d\" (UID: \"ca09d9b7-61ee-4cb1-8ac9-23595634fcae\") " pod="kuadrant-system/kuadrant-operator-catalog-s7n8d"
Jul 28 22:10:56.776757 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:56.776664    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-s7n8d"
Jul 28 22:10:56.832149 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:56.832105    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-s7n8d"]
Jul 28 22:10:56.907277 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:56.907219    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-s7n8d"]
Jul 28 22:10:56.910395 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:10:56.910366    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca09d9b7_61ee_4cb1_8ac9_23595634fcae.slice/crio-cf4acab193f153fb98e5cdaa1b63eff7111531bc7a11ab3b9c6dbfcc8be8d9c9 WatchSource:0}: Error finding container cf4acab193f153fb98e5cdaa1b63eff7111531bc7a11ab3b9c6dbfcc8be8d9c9: Status 404 returned error can't find the container with id cf4acab193f153fb98e5cdaa1b63eff7111531bc7a11ab3b9c6dbfcc8be8d9c9
Jul 28 22:10:56.970696 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:56.970657    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-s7n8d" event={"ID":"ca09d9b7-61ee-4cb1-8ac9-23595634fcae","Type":"ContainerStarted","Data":"cf4acab193f153fb98e5cdaa1b63eff7111531bc7a11ab3b9c6dbfcc8be8d9c9"}
Jul 28 22:10:57.039888 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:57.039801    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dzr7c"]
Jul 28 22:10:57.042514 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:57.042497    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dzr7c"
Jul 28 22:10:57.051195 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:57.051160    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dzr7c"]
Jul 28 22:10:57.167000 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:57.166966    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn9sv\" (UniqueName: \"kubernetes.io/projected/12d8cd98-7bc6-4991-b4c1-bd5ce9a4592b-kube-api-access-pn9sv\") pod \"kuadrant-operator-catalog-dzr7c\" (UID: \"12d8cd98-7bc6-4991-b4c1-bd5ce9a4592b\") " pod="kuadrant-system/kuadrant-operator-catalog-dzr7c"
Jul 28 22:10:57.267879 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:57.267834    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn9sv\" (UniqueName: \"kubernetes.io/projected/12d8cd98-7bc6-4991-b4c1-bd5ce9a4592b-kube-api-access-pn9sv\") pod \"kuadrant-operator-catalog-dzr7c\" (UID: \"12d8cd98-7bc6-4991-b4c1-bd5ce9a4592b\") " pod="kuadrant-system/kuadrant-operator-catalog-dzr7c"
Jul 28 22:10:57.275772 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:57.275742    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn9sv\" (UniqueName: \"kubernetes.io/projected/12d8cd98-7bc6-4991-b4c1-bd5ce9a4592b-kube-api-access-pn9sv\") pod \"kuadrant-operator-catalog-dzr7c\" (UID: \"12d8cd98-7bc6-4991-b4c1-bd5ce9a4592b\") " pod="kuadrant-system/kuadrant-operator-catalog-dzr7c"
Jul 28 22:10:57.352068 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:57.351987    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dzr7c"
Jul 28 22:10:57.505225 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:57.505127    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dzr7c"]
Jul 28 22:10:57.572476 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:10:57.572442    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12d8cd98_7bc6_4991_b4c1_bd5ce9a4592b.slice/crio-18d80b4a48faaeeda62b202b86c490aacb223da4e2450b3c09ec4cbe8da3c1c8 WatchSource:0}: Error finding container 18d80b4a48faaeeda62b202b86c490aacb223da4e2450b3c09ec4cbe8da3c1c8: Status 404 returned error can't find the container with id 18d80b4a48faaeeda62b202b86c490aacb223da4e2450b3c09ec4cbe8da3c1c8
Jul 28 22:10:57.978957 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:57.978915    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dzr7c" event={"ID":"12d8cd98-7bc6-4991-b4c1-bd5ce9a4592b","Type":"ContainerStarted","Data":"18d80b4a48faaeeda62b202b86c490aacb223da4e2450b3c09ec4cbe8da3c1c8"}
Jul 28 22:10:58.991454 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:58.991410    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dzr7c" event={"ID":"12d8cd98-7bc6-4991-b4c1-bd5ce9a4592b","Type":"ContainerStarted","Data":"1ba40c01c0db0da122328cc01d3e481def91f207e06f33ee5ff0679e9aad613b"}
Jul 28 22:10:58.992911 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:58.992874    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-s7n8d" event={"ID":"ca09d9b7-61ee-4cb1-8ac9-23595634fcae","Type":"ContainerStarted","Data":"367e473beac1c50f975a58a9269516c04526267e2ca181ae2e108fd646d91d3d"}
Jul 28 22:10:58.993025 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:58.992984    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-s7n8d" podUID="ca09d9b7-61ee-4cb1-8ac9-23595634fcae" containerName="registry-server" containerID="cri-o://367e473beac1c50f975a58a9269516c04526267e2ca181ae2e108fd646d91d3d" gracePeriod=2
Jul 28 22:10:59.005207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:59.005159    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-dzr7c" podStartSLOduration=0.79708842 podStartE2EDuration="2.005141174s" totalImagesPullingTime="1.208052754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:10:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:59.004037699 +0000 UTC m=+387.988180531" watchObservedRunningTime="2026-07-28 22:10:59.005141174 +0000 UTC m=+387.989283794"
Jul 28 22:10:59.017903 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:59.017858    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-s7n8d" podStartSLOduration=1.1505644959999999 podStartE2EDuration="3.017839729s" totalImagesPullingTime="1.867275233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:59.016798194 +0000 UTC m=+388.000940814" watchObservedRunningTime="2026-07-28 22:10:59.017839729 +0000 UTC m=+388.001982348"
Jul 28 22:10:59.235166 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:59.235137    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-s7n8d"
Jul 28 22:10:59.392410 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:59.392314    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4f5h\" (UniqueName: \"kubernetes.io/projected/ca09d9b7-61ee-4cb1-8ac9-23595634fcae-kube-api-access-p4f5h\") pod \"ca09d9b7-61ee-4cb1-8ac9-23595634fcae\" (UID: \"ca09d9b7-61ee-4cb1-8ac9-23595634fcae\") "
Jul 28 22:10:59.394558 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:59.394523    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca09d9b7-61ee-4cb1-8ac9-23595634fcae-kube-api-access-p4f5h" (OuterVolumeSpecName: "kube-api-access-p4f5h") pod "ca09d9b7-61ee-4cb1-8ac9-23595634fcae" (UID: "ca09d9b7-61ee-4cb1-8ac9-23595634fcae"). InnerVolumeSpecName "kube-api-access-p4f5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:10:59.493060 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:59.493026    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4f5h\" (UniqueName: \"kubernetes.io/projected/ca09d9b7-61ee-4cb1-8ac9-23595634fcae-kube-api-access-p4f5h\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:10:59.998192 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:59.998160    2574 generic.go:356] "Generic (PLEG): container finished" podID="ca09d9b7-61ee-4cb1-8ac9-23595634fcae" containerID="367e473beac1c50f975a58a9269516c04526267e2ca181ae2e108fd646d91d3d" exitCode=0
Jul 28 22:10:59.998686 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:59.998200    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-s7n8d"
Jul 28 22:10:59.998686 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:59.998260    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-s7n8d" event={"ID":"ca09d9b7-61ee-4cb1-8ac9-23595634fcae","Type":"ContainerDied","Data":"367e473beac1c50f975a58a9269516c04526267e2ca181ae2e108fd646d91d3d"}
Jul 28 22:10:59.998686 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:59.998312    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-s7n8d" event={"ID":"ca09d9b7-61ee-4cb1-8ac9-23595634fcae","Type":"ContainerDied","Data":"cf4acab193f153fb98e5cdaa1b63eff7111531bc7a11ab3b9c6dbfcc8be8d9c9"}
Jul 28 22:10:59.998686 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:10:59.998341    2574 scope.go:117] "RemoveContainer" containerID="367e473beac1c50f975a58a9269516c04526267e2ca181ae2e108fd646d91d3d"
Jul 28 22:11:00.007034 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:00.007014    2574 scope.go:117] "RemoveContainer" containerID="367e473beac1c50f975a58a9269516c04526267e2ca181ae2e108fd646d91d3d"
Jul 28 22:11:00.007336 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:11:00.007315    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"367e473beac1c50f975a58a9269516c04526267e2ca181ae2e108fd646d91d3d\": container with ID starting with 367e473beac1c50f975a58a9269516c04526267e2ca181ae2e108fd646d91d3d not found: ID does not exist" containerID="367e473beac1c50f975a58a9269516c04526267e2ca181ae2e108fd646d91d3d"
Jul 28 22:11:00.007387 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:00.007345    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"367e473beac1c50f975a58a9269516c04526267e2ca181ae2e108fd646d91d3d"} err="failed to get container status \"367e473beac1c50f975a58a9269516c04526267e2ca181ae2e108fd646d91d3d\": rpc error: code = NotFound desc = could not find container \"367e473beac1c50f975a58a9269516c04526267e2ca181ae2e108fd646d91d3d\": container with ID starting with 367e473beac1c50f975a58a9269516c04526267e2ca181ae2e108fd646d91d3d not found: ID does not exist"
Jul 28 22:11:00.011022 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:00.010998    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-s7n8d"]
Jul 28 22:11:00.017390 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:00.017365    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-s7n8d"]
Jul 28 22:11:01.492777 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:01.492736    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca09d9b7-61ee-4cb1-8ac9-23595634fcae" path="/var/lib/kubelet/pods/ca09d9b7-61ee-4cb1-8ac9-23595634fcae/volumes"
Jul 28 22:11:07.352989 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:07.352953    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-dzr7c"
Jul 28 22:11:07.353637 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:07.353021    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-dzr7c"
Jul 28 22:11:07.374951 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:07.374925    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-dzr7c"
Jul 28 22:11:08.053055 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:08.053027    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-dzr7c"
Jul 28 22:11:23.847083 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:23.847042    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-xf4vp"]
Jul 28 22:11:23.847717 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:23.847695    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca09d9b7-61ee-4cb1-8ac9-23595634fcae" containerName="registry-server"
Jul 28 22:11:23.847717 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:23.847719    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca09d9b7-61ee-4cb1-8ac9-23595634fcae" containerName="registry-server"
Jul 28 22:11:23.847871 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:23.847858    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca09d9b7-61ee-4cb1-8ac9-23595634fcae" containerName="registry-server"
Jul 28 22:11:23.851458 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:23.851432    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xf4vp"
Jul 28 22:11:23.853694 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:23.853668    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-48k8x\""
Jul 28 22:11:23.863073 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:23.863050    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-xf4vp"]
Jul 28 22:11:23.894695 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:23.894653    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhv6p\" (UniqueName: \"kubernetes.io/projected/ac4a98b1-7dd9-4269-ae2c-af949745aa15-kube-api-access-fhv6p\") pod \"authorino-operator-75cc8d5cb7-xf4vp\" (UID: \"ac4a98b1-7dd9-4269-ae2c-af949745aa15\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-xf4vp"
Jul 28 22:11:23.995681 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:23.995645    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhv6p\" (UniqueName: \"kubernetes.io/projected/ac4a98b1-7dd9-4269-ae2c-af949745aa15-kube-api-access-fhv6p\") pod \"authorino-operator-75cc8d5cb7-xf4vp\" (UID: \"ac4a98b1-7dd9-4269-ae2c-af949745aa15\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-xf4vp"
Jul 28 22:11:24.004536 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:24.004504    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhv6p\" (UniqueName: \"kubernetes.io/projected/ac4a98b1-7dd9-4269-ae2c-af949745aa15-kube-api-access-fhv6p\") pod \"authorino-operator-75cc8d5cb7-xf4vp\" (UID: \"ac4a98b1-7dd9-4269-ae2c-af949745aa15\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-xf4vp"
Jul 28 22:11:24.161405 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:24.161310    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xf4vp"
Jul 28 22:11:24.291926 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:24.291756    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-xf4vp"]
Jul 28 22:11:24.294699 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:11:24.294662    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac4a98b1_7dd9_4269_ae2c_af949745aa15.slice/crio-0e4de5543f0e56b310055f611d5d57028277e8121baa42581920a3d65b09a5e4 WatchSource:0}: Error finding container 0e4de5543f0e56b310055f611d5d57028277e8121baa42581920a3d65b09a5e4: Status 404 returned error can't find the container with id 0e4de5543f0e56b310055f611d5d57028277e8121baa42581920a3d65b09a5e4
Jul 28 22:11:25.104732 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:25.104700    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xf4vp" event={"ID":"ac4a98b1-7dd9-4269-ae2c-af949745aa15","Type":"ContainerStarted","Data":"0e4de5543f0e56b310055f611d5d57028277e8121baa42581920a3d65b09a5e4"}
Jul 28 22:11:26.977715 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:26.977678    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jsfk"]
Jul 28 22:11:26.979936 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:26.979919    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jsfk"
Jul 28 22:11:26.981650 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:26.981628    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 28 22:11:26.981650 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:26.981639    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-7t887\""
Jul 28 22:11:26.989816 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:26.989786    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jsfk"]
Jul 28 22:11:27.019071 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:27.019032    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrhbr\" (UniqueName: \"kubernetes.io/projected/46c2b945-8fe4-4ed0-9776-fd5336b32cda-kube-api-access-jrhbr\") pod \"dns-operator-controller-manager-864dd78c4c-8jsfk\" (UID: \"46c2b945-8fe4-4ed0-9776-fd5336b32cda\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jsfk"
Jul 28 22:11:27.114471 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:27.114427    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xf4vp" event={"ID":"ac4a98b1-7dd9-4269-ae2c-af949745aa15","Type":"ContainerStarted","Data":"93986687df5b4b82ec4705f58c663b8decfdcc3b3c373ddf9f1fda27dc842f10"}
Jul 28 22:11:27.114658 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:27.114519    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xf4vp"
Jul 28 22:11:27.120213 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:27.120182    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrhbr\" (UniqueName: \"kubernetes.io/projected/46c2b945-8fe4-4ed0-9776-fd5336b32cda-kube-api-access-jrhbr\") pod \"dns-operator-controller-manager-864dd78c4c-8jsfk\" (UID: \"46c2b945-8fe4-4ed0-9776-fd5336b32cda\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jsfk"
Jul 28 22:11:27.135494 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:27.135459    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrhbr\" (UniqueName: \"kubernetes.io/projected/46c2b945-8fe4-4ed0-9776-fd5336b32cda-kube-api-access-jrhbr\") pod \"dns-operator-controller-manager-864dd78c4c-8jsfk\" (UID: \"46c2b945-8fe4-4ed0-9776-fd5336b32cda\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jsfk"
Jul 28 22:11:27.291667 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:27.291576    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jsfk"
Jul 28 22:11:27.444430 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:27.444365    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xf4vp" podStartSLOduration=2.139103749 podStartE2EDuration="4.444340812s" totalImagesPullingTime="2.305237063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:11:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:11:27.151558033 +0000 UTC m=+416.135700652" watchObservedRunningTime="2026-07-28 22:11:27.444340812 +0000 UTC m=+416.428483435"
Jul 28 22:11:27.445346 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:27.445323    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jsfk"]
Jul 28 22:11:27.448803 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:11:27.448782    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46c2b945_8fe4_4ed0_9776_fd5336b32cda.slice/crio-f8667a385895007bc936610761a53b24a3b4df04c8dbe172128efe2cf3a4fcee WatchSource:0}: Error finding container f8667a385895007bc936610761a53b24a3b4df04c8dbe172128efe2cf3a4fcee: Status 404 returned error can't find the container with id f8667a385895007bc936610761a53b24a3b4df04c8dbe172128efe2cf3a4fcee
Jul 28 22:11:28.119986 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:28.119954    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jsfk" event={"ID":"46c2b945-8fe4-4ed0-9776-fd5336b32cda","Type":"ContainerStarted","Data":"f8667a385895007bc936610761a53b24a3b4df04c8dbe172128efe2cf3a4fcee"}
Jul 28 22:11:30.130883 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:30.130838    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jsfk" event={"ID":"46c2b945-8fe4-4ed0-9776-fd5336b32cda","Type":"ContainerStarted","Data":"cf043399f810f896bd6cfefd3c1f608943f7e6ebbec0acd754a2d03224ec14a5"}
Jul 28 22:11:30.131301 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:30.130977    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jsfk"
Jul 28 22:11:30.158589 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:30.158520    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jsfk" podStartSLOduration=2.282995343 podStartE2EDuration="4.158374145s" totalImagesPullingTime="1.875378802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:11:30.157440492 +0000 UTC m=+419.141583092" watchObservedRunningTime="2026-07-28 22:11:30.158374145 +0000 UTC m=+419.142516770"
Jul 28 22:11:31.647634 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:31.647551    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"]
Jul 28 22:11:31.655186 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:31.655160    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"
Jul 28 22:11:31.657200 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:31.657180    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-5fr88\""
Jul 28 22:11:31.663397 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:31.663374    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"]
Jul 28 22:11:31.761341 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:31.761306    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d40f7b35-3950-4b07-9c0e-594809295123-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prtl5\" (UID: \"d40f7b35-3950-4b07-9c0e-594809295123\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"
Jul 28 22:11:31.761341 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:31.761343    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsbvw\" (UniqueName: \"kubernetes.io/projected/d40f7b35-3950-4b07-9c0e-594809295123-kube-api-access-lsbvw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prtl5\" (UID: \"d40f7b35-3950-4b07-9c0e-594809295123\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"
Jul 28 22:11:31.861804 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:31.861760    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d40f7b35-3950-4b07-9c0e-594809295123-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prtl5\" (UID: \"d40f7b35-3950-4b07-9c0e-594809295123\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"
Jul 28 22:11:31.861804 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:31.861805    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsbvw\" (UniqueName: \"kubernetes.io/projected/d40f7b35-3950-4b07-9c0e-594809295123-kube-api-access-lsbvw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prtl5\" (UID: \"d40f7b35-3950-4b07-9c0e-594809295123\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"
Jul 28 22:11:31.862215 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:31.862190    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d40f7b35-3950-4b07-9c0e-594809295123-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prtl5\" (UID: \"d40f7b35-3950-4b07-9c0e-594809295123\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"
Jul 28 22:11:31.876748 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:31.876710    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsbvw\" (UniqueName: \"kubernetes.io/projected/d40f7b35-3950-4b07-9c0e-594809295123-kube-api-access-lsbvw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prtl5\" (UID: \"d40f7b35-3950-4b07-9c0e-594809295123\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"
Jul 28 22:11:31.964827 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:31.964781    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"
Jul 28 22:11:32.132021 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:32.131979    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"]
Jul 28 22:11:32.139369 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:11:32.139341    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd40f7b35_3950_4b07_9c0e_594809295123.slice/crio-a2267eb1ee54afbc2ce771ca4da64de059b7834ef84ee98a907e74f8c01a0bbb WatchSource:0}: Error finding container a2267eb1ee54afbc2ce771ca4da64de059b7834ef84ee98a907e74f8c01a0bbb: Status 404 returned error can't find the container with id a2267eb1ee54afbc2ce771ca4da64de059b7834ef84ee98a907e74f8c01a0bbb
Jul 28 22:11:33.145630 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:33.145586    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5" event={"ID":"d40f7b35-3950-4b07-9c0e-594809295123","Type":"ContainerStarted","Data":"a2267eb1ee54afbc2ce771ca4da64de059b7834ef84ee98a907e74f8c01a0bbb"}
Jul 28 22:11:33.808999 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:33.808960    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff"]
Jul 28 22:11:33.812546 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:33.812523    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff"
Jul 28 22:11:33.814409 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:33.814393    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-n59n8\""
Jul 28 22:11:33.822985 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:33.822958    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff"]
Jul 28 22:11:33.878075 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:33.878035    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpmrd\" (UniqueName: \"kubernetes.io/projected/02f9596f-9b1d-4d51-b9af-6d7a2f8c5169-kube-api-access-cpmrd\") pod \"limitador-operator-controller-manager-859d7b78bd-5kzff\" (UID: \"02f9596f-9b1d-4d51-b9af-6d7a2f8c5169\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff"
Jul 28 22:11:33.979053 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:33.979012    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpmrd\" (UniqueName: \"kubernetes.io/projected/02f9596f-9b1d-4d51-b9af-6d7a2f8c5169-kube-api-access-cpmrd\") pod \"limitador-operator-controller-manager-859d7b78bd-5kzff\" (UID: \"02f9596f-9b1d-4d51-b9af-6d7a2f8c5169\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff"
Jul 28 22:11:33.989808 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:33.989776    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpmrd\" (UniqueName: \"kubernetes.io/projected/02f9596f-9b1d-4d51-b9af-6d7a2f8c5169-kube-api-access-cpmrd\") pod \"limitador-operator-controller-manager-859d7b78bd-5kzff\" (UID: \"02f9596f-9b1d-4d51-b9af-6d7a2f8c5169\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff"
Jul 28 22:11:34.124256 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:34.124144    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff"
Jul 28 22:11:34.275495 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:34.275457    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff"]
Jul 28 22:11:34.279828 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:11:34.279786    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02f9596f_9b1d_4d51_b9af_6d7a2f8c5169.slice/crio-fbfe0759c63c6d5966c55a3642ab6d9cac5712c056fe09cbd9a889331a5ac270 WatchSource:0}: Error finding container fbfe0759c63c6d5966c55a3642ab6d9cac5712c056fe09cbd9a889331a5ac270: Status 404 returned error can't find the container with id fbfe0759c63c6d5966c55a3642ab6d9cac5712c056fe09cbd9a889331a5ac270
Jul 28 22:11:35.158914 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:35.158871    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff" event={"ID":"02f9596f-9b1d-4d51-b9af-6d7a2f8c5169","Type":"ContainerStarted","Data":"fbfe0759c63c6d5966c55a3642ab6d9cac5712c056fe09cbd9a889331a5ac270"}
Jul 28 22:11:36.165951 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:36.165854    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff" event={"ID":"02f9596f-9b1d-4d51-b9af-6d7a2f8c5169","Type":"ContainerStarted","Data":"e95edf020c619a97fc6f18c3b786bffdb62c298e4e8dfda0aa7b4b1bcd6e02b4"}
Jul 28 22:11:36.166435 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:36.165995    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff"
Jul 28 22:11:36.181345 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:36.181285    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff" podStartSLOduration=1.5838904440000001 podStartE2EDuration="3.181268223s" totalImagesPullingTime="1.597377779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:11:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:11:36.180385407 +0000 UTC m=+425.164528039" watchObservedRunningTime="2026-07-28 22:11:36.181268223 +0000 UTC m=+425.165410842"
Jul 28 22:11:38.122862 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:38.122830    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xf4vp"
Jul 28 22:11:38.177335 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:38.177299    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5" event={"ID":"d40f7b35-3950-4b07-9c0e-594809295123","Type":"ContainerStarted","Data":"f4ff74b767639864a6a3a9caddb09e091340c0b234e1e53e5b6916bcf1366cce"}
Jul 28 22:11:38.177516 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:38.177433    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"
Jul 28 22:11:38.214129 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:38.214067    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5" podStartSLOduration=2.071799112 podStartE2EDuration="7.214048729s" totalImagesPullingTime="5.142249617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:11:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:11:38.213929901 +0000 UTC m=+427.198072557" watchObservedRunningTime="2026-07-28 22:11:38.214048729 +0000 UTC m=+427.198191349"
Jul 28 22:11:41.137930 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:41.137894    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jsfk"
Jul 28 22:11:47.174062 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:47.174032    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff"
Jul 28 22:11:49.184531 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:49.184497    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"
Jul 28 22:11:50.948060 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:50.948025    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"]
Jul 28 22:11:50.948532 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:50.948215    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5" podUID="d40f7b35-3950-4b07-9c0e-594809295123" containerName="manager" containerID="cri-o://f4ff74b767639864a6a3a9caddb09e091340c0b234e1e53e5b6916bcf1366cce" gracePeriod=2
Jul 28 22:11:50.954297 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:50.954267    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"]
Jul 28 22:11:50.960033 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:11:50.959997    2574 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d40f7b35-3950-4b07-9c0e-594809295123\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-prtl5\": pods \"kuadrant-operator-controller-manager-8684c9ff49-prtl5\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"
Jul 28 22:11:50.974223 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:50.974192    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff"]
Jul 28 22:11:50.974500 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:50.974450    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff" podUID="02f9596f-9b1d-4d51-b9af-6d7a2f8c5169" containerName="manager" containerID="cri-o://e95edf020c619a97fc6f18c3b786bffdb62c298e4e8dfda0aa7b4b1bcd6e02b4" gracePeriod=2
Jul 28 22:11:50.976102 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:11:50.976064    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prtl5\" is forbidden: User \"system:node:ip-10-0-143-69.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-69.ec2.internal' and this object" podUID="d40f7b35-3950-4b07-9c0e-594809295123" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"
Jul 28 22:11:50.976439 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:50.976411    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24"]
Jul 28 22:11:50.976878 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:50.976857    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d40f7b35-3950-4b07-9c0e-594809295123" containerName="manager"
Jul 28 22:11:50.976991 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:50.976879    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d40f7b35-3950-4b07-9c0e-594809295123" containerName="manager"
Jul 28 22:11:50.976991 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:50.976976    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d40f7b35-3950-4b07-9c0e-594809295123" containerName="manager"
Jul 28 22:11:50.980048 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:50.980027    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24"
Jul 28 22:11:50.987932 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:50.987899    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff"]
Jul 28 22:11:50.999173 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:50.999119    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24"]
Jul 28 22:11:51.000997 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.000971    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4ns4b"]
Jul 28 22:11:51.001307 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.001293    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02f9596f-9b1d-4d51-b9af-6d7a2f8c5169" containerName="manager"
Jul 28 22:11:51.001358 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.001308    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="02f9596f-9b1d-4d51-b9af-6d7a2f8c5169" containerName="manager"
Jul 28 22:11:51.001358 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.001359    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="02f9596f-9b1d-4d51-b9af-6d7a2f8c5169" containerName="manager"
Jul 28 22:11:51.004362 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.004344    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4ns4b"
Jul 28 22:11:51.006399 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:11:51.006371    2574 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"02f9596f-9b1d-4d51-b9af-6d7a2f8c5169\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-5kzff\": pods \"limitador-operator-controller-manager-859d7b78bd-5kzff\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff"
Jul 28 22:11:51.007951 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:11:51.007921    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prtl5\" is forbidden: User \"system:node:ip-10-0-143-69.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-69.ec2.internal' and this object" podUID="d40f7b35-3950-4b07-9c0e-594809295123" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"
Jul 28 22:11:51.009312 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:11:51.009286    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5kzff\" is forbidden: User \"system:node:ip-10-0-143-69.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-69.ec2.internal' and this object" podUID="02f9596f-9b1d-4d51-b9af-6d7a2f8c5169" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff"
Jul 28 22:11:51.011769 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.011608    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4ns4b"]
Jul 28 22:11:51.125000 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.124967    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b52f04d0-d424-46b8-b4b5-5ed165345263-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j8g24\" (UID: \"b52f04d0-d424-46b8-b4b5-5ed165345263\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24"
Jul 28 22:11:51.125171 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.125023    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptfd5\" (UniqueName: \"kubernetes.io/projected/8559d00a-827c-4feb-ad81-b1e3f7fcdedc-kube-api-access-ptfd5\") pod \"limitador-operator-controller-manager-859d7b78bd-4ns4b\" (UID: \"8559d00a-827c-4feb-ad81-b1e3f7fcdedc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4ns4b"
Jul 28 22:11:51.125171 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.125054    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x5f8\" (UniqueName: \"kubernetes.io/projected/b52f04d0-d424-46b8-b4b5-5ed165345263-kube-api-access-5x5f8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j8g24\" (UID: \"b52f04d0-d424-46b8-b4b5-5ed165345263\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24"
Jul 28 22:11:51.225733 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.225701    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b52f04d0-d424-46b8-b4b5-5ed165345263-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j8g24\" (UID: \"b52f04d0-d424-46b8-b4b5-5ed165345263\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24"
Jul 28 22:11:51.225907 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.225778    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptfd5\" (UniqueName: \"kubernetes.io/projected/8559d00a-827c-4feb-ad81-b1e3f7fcdedc-kube-api-access-ptfd5\") pod \"limitador-operator-controller-manager-859d7b78bd-4ns4b\" (UID: \"8559d00a-827c-4feb-ad81-b1e3f7fcdedc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4ns4b"
Jul 28 22:11:51.225907 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.225797    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x5f8\" (UniqueName: \"kubernetes.io/projected/b52f04d0-d424-46b8-b4b5-5ed165345263-kube-api-access-5x5f8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j8g24\" (UID: \"b52f04d0-d424-46b8-b4b5-5ed165345263\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24"
Jul 28 22:11:51.226080 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.226057    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b52f04d0-d424-46b8-b4b5-5ed165345263-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j8g24\" (UID: \"b52f04d0-d424-46b8-b4b5-5ed165345263\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24"
Jul 28 22:11:51.226453 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.226435    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"
Jul 28 22:11:51.228009 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:11:51.227981    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prtl5\" is forbidden: User \"system:node:ip-10-0-143-69.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-69.ec2.internal' and this object" podUID="d40f7b35-3950-4b07-9c0e-594809295123" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"
Jul 28 22:11:51.229086 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.229066    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff"
Jul 28 22:11:51.229488 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:11:51.229466    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5kzff\" is forbidden: User \"system:node:ip-10-0-143-69.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-69.ec2.internal' and this object" podUID="02f9596f-9b1d-4d51-b9af-6d7a2f8c5169" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff"
Jul 28 22:11:51.230921 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:11:51.230895    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prtl5\" is forbidden: User \"system:node:ip-10-0-143-69.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-69.ec2.internal' and this object" podUID="d40f7b35-3950-4b07-9c0e-594809295123" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"
Jul 28 22:11:51.232331 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:11:51.232297    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5kzff\" is forbidden: User \"system:node:ip-10-0-143-69.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-69.ec2.internal' and this object" podUID="02f9596f-9b1d-4d51-b9af-6d7a2f8c5169" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff"
Jul 28 22:11:51.233876 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.233853    2574 generic.go:356] "Generic (PLEG): container finished" podID="d40f7b35-3950-4b07-9c0e-594809295123" containerID="f4ff74b767639864a6a3a9caddb09e091340c0b234e1e53e5b6916bcf1366cce" exitCode=0
Jul 28 22:11:51.233957 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.233903    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"
Jul 28 22:11:51.233957 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.233948    2574 scope.go:117] "RemoveContainer" containerID="f4ff74b767639864a6a3a9caddb09e091340c0b234e1e53e5b6916bcf1366cce"
Jul 28 22:11:51.234049 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.233994    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x5f8\" (UniqueName: \"kubernetes.io/projected/b52f04d0-d424-46b8-b4b5-5ed165345263-kube-api-access-5x5f8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j8g24\" (UID: \"b52f04d0-d424-46b8-b4b5-5ed165345263\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24"
Jul 28 22:11:51.234049 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.234015    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptfd5\" (UniqueName: \"kubernetes.io/projected/8559d00a-827c-4feb-ad81-b1e3f7fcdedc-kube-api-access-ptfd5\") pod \"limitador-operator-controller-manager-859d7b78bd-4ns4b\" (UID: \"8559d00a-827c-4feb-ad81-b1e3f7fcdedc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4ns4b"
Jul 28 22:11:51.235222 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.235204    2574 generic.go:356] "Generic (PLEG): container finished" podID="02f9596f-9b1d-4d51-b9af-6d7a2f8c5169" containerID="e95edf020c619a97fc6f18c3b786bffdb62c298e4e8dfda0aa7b4b1bcd6e02b4" exitCode=0
Jul 28 22:11:51.235335 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.235272    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff"
Jul 28 22:11:51.235418 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:11:51.235400    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prtl5\" is forbidden: User \"system:node:ip-10-0-143-69.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-69.ec2.internal' and this object" podUID="d40f7b35-3950-4b07-9c0e-594809295123" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"
Jul 28 22:11:51.237105 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:11:51.237083    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5kzff\" is forbidden: User \"system:node:ip-10-0-143-69.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-69.ec2.internal' and this object" podUID="02f9596f-9b1d-4d51-b9af-6d7a2f8c5169" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff"
Jul 28 22:11:51.238643 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:11:51.238612    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prtl5\" is forbidden: User \"system:node:ip-10-0-143-69.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-69.ec2.internal' and this object" podUID="d40f7b35-3950-4b07-9c0e-594809295123" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"
Jul 28 22:11:51.240128 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:11:51.240108    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5kzff\" is forbidden: User \"system:node:ip-10-0-143-69.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-69.ec2.internal' and this object" podUID="02f9596f-9b1d-4d51-b9af-6d7a2f8c5169" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff"
Jul 28 22:11:51.242128 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.242109    2574 scope.go:117] "RemoveContainer" containerID="f4ff74b767639864a6a3a9caddb09e091340c0b234e1e53e5b6916bcf1366cce"
Jul 28 22:11:51.242427 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:11:51.242410    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4ff74b767639864a6a3a9caddb09e091340c0b234e1e53e5b6916bcf1366cce\": container with ID starting with f4ff74b767639864a6a3a9caddb09e091340c0b234e1e53e5b6916bcf1366cce not found: ID does not exist" containerID="f4ff74b767639864a6a3a9caddb09e091340c0b234e1e53e5b6916bcf1366cce"
Jul 28 22:11:51.242507 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.242443    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4ff74b767639864a6a3a9caddb09e091340c0b234e1e53e5b6916bcf1366cce"} err="failed to get container status \"f4ff74b767639864a6a3a9caddb09e091340c0b234e1e53e5b6916bcf1366cce\": rpc error: code = NotFound desc = could not find container \"f4ff74b767639864a6a3a9caddb09e091340c0b234e1e53e5b6916bcf1366cce\": container with ID starting with f4ff74b767639864a6a3a9caddb09e091340c0b234e1e53e5b6916bcf1366cce not found: ID does not exist"
Jul 28 22:11:51.242507 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.242466    2574 scope.go:117] "RemoveContainer" containerID="e95edf020c619a97fc6f18c3b786bffdb62c298e4e8dfda0aa7b4b1bcd6e02b4"
Jul 28 22:11:51.250110 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.250090    2574 scope.go:117] "RemoveContainer" containerID="e95edf020c619a97fc6f18c3b786bffdb62c298e4e8dfda0aa7b4b1bcd6e02b4"
Jul 28 22:11:51.250413 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:11:51.250395    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e95edf020c619a97fc6f18c3b786bffdb62c298e4e8dfda0aa7b4b1bcd6e02b4\": container with ID starting with e95edf020c619a97fc6f18c3b786bffdb62c298e4e8dfda0aa7b4b1bcd6e02b4 not found: ID does not exist" containerID="e95edf020c619a97fc6f18c3b786bffdb62c298e4e8dfda0aa7b4b1bcd6e02b4"
Jul 28 22:11:51.250498 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.250423    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e95edf020c619a97fc6f18c3b786bffdb62c298e4e8dfda0aa7b4b1bcd6e02b4"} err="failed to get container status \"e95edf020c619a97fc6f18c3b786bffdb62c298e4e8dfda0aa7b4b1bcd6e02b4\": rpc error: code = NotFound desc = could not find container \"e95edf020c619a97fc6f18c3b786bffdb62c298e4e8dfda0aa7b4b1bcd6e02b4\": container with ID starting with e95edf020c619a97fc6f18c3b786bffdb62c298e4e8dfda0aa7b4b1bcd6e02b4 not found: ID does not exist"
Jul 28 22:11:51.326669 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.326634    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpmrd\" (UniqueName: \"kubernetes.io/projected/02f9596f-9b1d-4d51-b9af-6d7a2f8c5169-kube-api-access-cpmrd\") pod \"02f9596f-9b1d-4d51-b9af-6d7a2f8c5169\" (UID: \"02f9596f-9b1d-4d51-b9af-6d7a2f8c5169\") "
Jul 28 22:11:51.326821 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.326704    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsbvw\" (UniqueName: \"kubernetes.io/projected/d40f7b35-3950-4b07-9c0e-594809295123-kube-api-access-lsbvw\") pod \"d40f7b35-3950-4b07-9c0e-594809295123\" (UID: \"d40f7b35-3950-4b07-9c0e-594809295123\") "
Jul 28 22:11:51.326883 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.326815    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d40f7b35-3950-4b07-9c0e-594809295123-extensions-socket-volume\") pod \"d40f7b35-3950-4b07-9c0e-594809295123\" (UID: \"d40f7b35-3950-4b07-9c0e-594809295123\") "
Jul 28 22:11:51.327343 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.327322    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d40f7b35-3950-4b07-9c0e-594809295123-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d40f7b35-3950-4b07-9c0e-594809295123" (UID: "d40f7b35-3950-4b07-9c0e-594809295123"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:11:51.328897 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.328878    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d40f7b35-3950-4b07-9c0e-594809295123-kube-api-access-lsbvw" (OuterVolumeSpecName: "kube-api-access-lsbvw") pod "d40f7b35-3950-4b07-9c0e-594809295123" (UID: "d40f7b35-3950-4b07-9c0e-594809295123"). InnerVolumeSpecName "kube-api-access-lsbvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:11:51.328965 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.328898    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02f9596f-9b1d-4d51-b9af-6d7a2f8c5169-kube-api-access-cpmrd" (OuterVolumeSpecName: "kube-api-access-cpmrd") pod "02f9596f-9b1d-4d51-b9af-6d7a2f8c5169" (UID: "02f9596f-9b1d-4d51-b9af-6d7a2f8c5169"). InnerVolumeSpecName "kube-api-access-cpmrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:11:51.395256 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.395195    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24"
Jul 28 22:11:51.402803 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.402777    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4ns4b"
Jul 28 22:11:51.428097 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.428067    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsbvw\" (UniqueName: \"kubernetes.io/projected/d40f7b35-3950-4b07-9c0e-594809295123-kube-api-access-lsbvw\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:11:51.428097 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.428093    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d40f7b35-3950-4b07-9c0e-594809295123-extensions-socket-volume\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:11:51.428310 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.428109    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpmrd\" (UniqueName: \"kubernetes.io/projected/02f9596f-9b1d-4d51-b9af-6d7a2f8c5169-kube-api-access-cpmrd\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:11:51.492893 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:11:51.492845    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prtl5\" is forbidden: User \"system:node:ip-10-0-143-69.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-69.ec2.internal' and this object" podUID="d40f7b35-3950-4b07-9c0e-594809295123" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prtl5"
Jul 28 22:11:51.494691 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:11:51.494626    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5kzff\" is forbidden: User \"system:node:ip-10-0-143-69.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-69.ec2.internal' and this object" podUID="02f9596f-9b1d-4d51-b9af-6d7a2f8c5169" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5kzff"
Jul 28 22:11:51.494845 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.494712    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02f9596f-9b1d-4d51-b9af-6d7a2f8c5169" path="/var/lib/kubelet/pods/02f9596f-9b1d-4d51-b9af-6d7a2f8c5169/volumes"
Jul 28 22:11:51.495178 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.495156    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d40f7b35-3950-4b07-9c0e-594809295123" path="/var/lib/kubelet/pods/d40f7b35-3950-4b07-9c0e-594809295123/volumes"
Jul 28 22:11:51.548549 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.548523    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24"]
Jul 28 22:11:51.550615 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:11:51.550586    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb52f04d0_d424_46b8_b4b5_5ed165345263.slice/crio-695a820d001a046ec35d40b2c3cbe89c73f7bb82d147c5030de3caa28c920e1d WatchSource:0}: Error finding container 695a820d001a046ec35d40b2c3cbe89c73f7bb82d147c5030de3caa28c920e1d: Status 404 returned error can't find the container with id 695a820d001a046ec35d40b2c3cbe89c73f7bb82d147c5030de3caa28c920e1d
Jul 28 22:11:51.560373 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:51.560350    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4ns4b"]
Jul 28 22:11:51.562796 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:11:51.562771    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8559d00a_827c_4feb_ad81_b1e3f7fcdedc.slice/crio-355343a3390518da766f2d0343d97f9ec569d73a4aa58754b782bdd496c1a238 WatchSource:0}: Error finding container 355343a3390518da766f2d0343d97f9ec569d73a4aa58754b782bdd496c1a238: Status 404 returned error can't find the container with id 355343a3390518da766f2d0343d97f9ec569d73a4aa58754b782bdd496c1a238
Jul 28 22:11:52.245015 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:52.244905    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4ns4b" event={"ID":"8559d00a-827c-4feb-ad81-b1e3f7fcdedc","Type":"ContainerStarted","Data":"edc3a02e1e3432f9a7c5d160d7c5b4a94efca59cbd3fc63d474069923f761dc9"}
Jul 28 22:11:52.245015 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:52.244974    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4ns4b" event={"ID":"8559d00a-827c-4feb-ad81-b1e3f7fcdedc","Type":"ContainerStarted","Data":"355343a3390518da766f2d0343d97f9ec569d73a4aa58754b782bdd496c1a238"}
Jul 28 22:11:52.245506 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:52.245102    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4ns4b"
Jul 28 22:11:52.246823 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:52.246795    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24" event={"ID":"b52f04d0-d424-46b8-b4b5-5ed165345263","Type":"ContainerStarted","Data":"991a494e2213927dde046e91ae445e958b2a62d85056635f9d1861e1fde8bf7c"}
Jul 28 22:11:52.246965 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:52.246837    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24" event={"ID":"b52f04d0-d424-46b8-b4b5-5ed165345263","Type":"ContainerStarted","Data":"695a820d001a046ec35d40b2c3cbe89c73f7bb82d147c5030de3caa28c920e1d"}
Jul 28 22:11:52.246965 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:52.246881    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24"
Jul 28 22:11:52.263803 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:52.263749    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4ns4b" podStartSLOduration=2.263734381 podStartE2EDuration="2.263734381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:11:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:11:52.262318426 +0000 UTC m=+441.246461046" watchObservedRunningTime="2026-07-28 22:11:52.263734381 +0000 UTC m=+441.247876978"
Jul 28 22:11:52.285287 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:11:52.285219    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24" podStartSLOduration=2.2852016 podStartE2EDuration="2.2852016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:11:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:11:52.284371245 +0000 UTC m=+441.268513888" watchObservedRunningTime="2026-07-28 22:11:52.2852016 +0000 UTC m=+441.269344219"
Jul 28 22:12:03.254206 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:03.254125    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4ns4b"
Jul 28 22:12:03.254690 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:03.254207    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24"
Jul 28 22:12:07.179399 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:07.179363    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24"]
Jul 28 22:12:07.179790 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:07.179578    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24" podUID="b52f04d0-d424-46b8-b4b5-5ed165345263" containerName="manager" containerID="cri-o://991a494e2213927dde046e91ae445e958b2a62d85056635f9d1861e1fde8bf7c" gracePeriod=10
Jul 28 22:12:07.316805 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:07.316765    2574 generic.go:356] "Generic (PLEG): container finished" podID="b52f04d0-d424-46b8-b4b5-5ed165345263" containerID="991a494e2213927dde046e91ae445e958b2a62d85056635f9d1861e1fde8bf7c" exitCode=0
Jul 28 22:12:07.316991 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:07.316828    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24" event={"ID":"b52f04d0-d424-46b8-b4b5-5ed165345263","Type":"ContainerDied","Data":"991a494e2213927dde046e91ae445e958b2a62d85056635f9d1861e1fde8bf7c"}
Jul 28 22:12:07.395194 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:07.395159    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6"]
Jul 28 22:12:07.398746 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:07.398720    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6"
Jul 28 22:12:07.411725 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:07.411695    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6"]
Jul 28 22:12:07.438451 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:07.438392    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24"
Jul 28 22:12:07.478349 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:07.478300    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xnvw\" (UniqueName: \"kubernetes.io/projected/5cbe3294-4936-496e-bc0d-1cc0c7e1e876-kube-api-access-5xnvw\") pod \"kuadrant-operator-controller-manager-86878c769f-65vl6\" (UID: \"5cbe3294-4936-496e-bc0d-1cc0c7e1e876\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6"
Jul 28 22:12:07.478542 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:07.478478    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cbe3294-4936-496e-bc0d-1cc0c7e1e876-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-65vl6\" (UID: \"5cbe3294-4936-496e-bc0d-1cc0c7e1e876\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6"
Jul 28 22:12:07.579667 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:07.579629    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b52f04d0-d424-46b8-b4b5-5ed165345263-extensions-socket-volume\") pod \"b52f04d0-d424-46b8-b4b5-5ed165345263\" (UID: \"b52f04d0-d424-46b8-b4b5-5ed165345263\") "
Jul 28 22:12:07.579867 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:07.579708    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5x5f8\" (UniqueName: \"kubernetes.io/projected/b52f04d0-d424-46b8-b4b5-5ed165345263-kube-api-access-5x5f8\") pod \"b52f04d0-d424-46b8-b4b5-5ed165345263\" (UID: \"b52f04d0-d424-46b8-b4b5-5ed165345263\") "
Jul 28 22:12:07.579867 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:07.579853    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xnvw\" (UniqueName: \"kubernetes.io/projected/5cbe3294-4936-496e-bc0d-1cc0c7e1e876-kube-api-access-5xnvw\") pod \"kuadrant-operator-controller-manager-86878c769f-65vl6\" (UID: \"5cbe3294-4936-496e-bc0d-1cc0c7e1e876\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6"
Jul 28 22:12:07.579990 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:07.579938    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cbe3294-4936-496e-bc0d-1cc0c7e1e876-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-65vl6\" (UID: \"5cbe3294-4936-496e-bc0d-1cc0c7e1e876\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6"
Jul 28 22:12:07.580049 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:07.580016    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b52f04d0-d424-46b8-b4b5-5ed165345263-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b52f04d0-d424-46b8-b4b5-5ed165345263" (UID: "b52f04d0-d424-46b8-b4b5-5ed165345263"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:12:07.580320 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:07.580297    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cbe3294-4936-496e-bc0d-1cc0c7e1e876-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-65vl6\" (UID: \"5cbe3294-4936-496e-bc0d-1cc0c7e1e876\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6"
Jul 28 22:12:07.581835 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:07.581802    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b52f04d0-d424-46b8-b4b5-5ed165345263-kube-api-access-5x5f8" (OuterVolumeSpecName: "kube-api-access-5x5f8") pod "b52f04d0-d424-46b8-b4b5-5ed165345263" (UID: "b52f04d0-d424-46b8-b4b5-5ed165345263"). InnerVolumeSpecName "kube-api-access-5x5f8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:12:07.589147 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:07.589116    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xnvw\" (UniqueName: \"kubernetes.io/projected/5cbe3294-4936-496e-bc0d-1cc0c7e1e876-kube-api-access-5xnvw\") pod \"kuadrant-operator-controller-manager-86878c769f-65vl6\" (UID: \"5cbe3294-4936-496e-bc0d-1cc0c7e1e876\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6"
Jul 28 22:12:07.680870 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:07.680826    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b52f04d0-d424-46b8-b4b5-5ed165345263-extensions-socket-volume\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:12:07.680870 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:07.680860    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5x5f8\" (UniqueName: \"kubernetes.io/projected/b52f04d0-d424-46b8-b4b5-5ed165345263-kube-api-access-5x5f8\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:12:07.708870 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:07.708833    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6"
Jul 28 22:12:07.845416 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:07.845384    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6"]
Jul 28 22:12:07.848735 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:12:07.848705    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cbe3294_4936_496e_bc0d_1cc0c7e1e876.slice/crio-0ff63eff59bd89a558f292aece436793a07d23b816b6e5f0c48fbc214a7aba83 WatchSource:0}: Error finding container 0ff63eff59bd89a558f292aece436793a07d23b816b6e5f0c48fbc214a7aba83: Status 404 returned error can't find the container with id 0ff63eff59bd89a558f292aece436793a07d23b816b6e5f0c48fbc214a7aba83
Jul 28 22:12:08.322261 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:08.322212    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24"
Jul 28 22:12:08.322261 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:08.322219    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24" event={"ID":"b52f04d0-d424-46b8-b4b5-5ed165345263","Type":"ContainerDied","Data":"695a820d001a046ec35d40b2c3cbe89c73f7bb82d147c5030de3caa28c920e1d"}
Jul 28 22:12:08.322786 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:08.322315    2574 scope.go:117] "RemoveContainer" containerID="991a494e2213927dde046e91ae445e958b2a62d85056635f9d1861e1fde8bf7c"
Jul 28 22:12:08.324181 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:08.324160    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6" event={"ID":"5cbe3294-4936-496e-bc0d-1cc0c7e1e876","Type":"ContainerStarted","Data":"5638f09ff70100b281bd87ce6fc2ab2a89736b39d837c2711ed609ab5ec78909"}
Jul 28 22:12:08.324315 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:08.324210    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6" event={"ID":"5cbe3294-4936-496e-bc0d-1cc0c7e1e876","Type":"ContainerStarted","Data":"0ff63eff59bd89a558f292aece436793a07d23b816b6e5f0c48fbc214a7aba83"}
Jul 28 22:12:08.324315 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:08.324267    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6"
Jul 28 22:12:08.343746 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:08.343694    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6" podStartSLOduration=1.343676944 podStartE2EDuration="1.343676944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:12:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:12:08.342574213 +0000 UTC m=+457.326716835" watchObservedRunningTime="2026-07-28 22:12:08.343676944 +0000 UTC m=+457.327819564"
Jul 28 22:12:08.363099 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:08.363064    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24"]
Jul 28 22:12:08.366963 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:08.366937    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j8g24"]
Jul 28 22:12:09.497756 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:09.497719    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b52f04d0-d424-46b8-b4b5-5ed165345263" path="/var/lib/kubelet/pods/b52f04d0-d424-46b8-b4b5-5ed165345263/volumes"
Jul 28 22:12:19.332585 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:19.332552    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6"
Jul 28 22:12:23.592801 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.592766    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"]
Jul 28 22:12:23.593175 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.593071    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b52f04d0-d424-46b8-b4b5-5ed165345263" containerName="manager"
Jul 28 22:12:23.593175 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.593081    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b52f04d0-d424-46b8-b4b5-5ed165345263" containerName="manager"
Jul 28 22:12:23.593175 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.593142    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b52f04d0-d424-46b8-b4b5-5ed165345263" containerName="manager"
Jul 28 22:12:23.596193 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.596174    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.598966 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.598944    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-nxtch\""
Jul 28 22:12:23.610035 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.610009    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"]
Jul 28 22:12:23.693084 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.693041    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a55b9c60-64aa-4a57-82d7-6309caa50034-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.693084 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.693077    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a55b9c60-64aa-4a57-82d7-6309caa50034-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.693391 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.693104    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a55b9c60-64aa-4a57-82d7-6309caa50034-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.693391 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.693155    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhh6q\" (UniqueName: \"kubernetes.io/projected/a55b9c60-64aa-4a57-82d7-6309caa50034-kube-api-access-bhh6q\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.693391 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.693201    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a55b9c60-64aa-4a57-82d7-6309caa50034-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.693391 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.693266    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a55b9c60-64aa-4a57-82d7-6309caa50034-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.693391 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.693320    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a55b9c60-64aa-4a57-82d7-6309caa50034-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.693391 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.693355    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a55b9c60-64aa-4a57-82d7-6309caa50034-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.693391 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.693380    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a55b9c60-64aa-4a57-82d7-6309caa50034-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.794659 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.794618    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a55b9c60-64aa-4a57-82d7-6309caa50034-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.794659 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.794659    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a55b9c60-64aa-4a57-82d7-6309caa50034-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.794930 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.794692    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a55b9c60-64aa-4a57-82d7-6309caa50034-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.794930 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.794716    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhh6q\" (UniqueName: \"kubernetes.io/projected/a55b9c60-64aa-4a57-82d7-6309caa50034-kube-api-access-bhh6q\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.795092 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.794749    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a55b9c60-64aa-4a57-82d7-6309caa50034-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.795189 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.795119    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a55b9c60-64aa-4a57-82d7-6309caa50034-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.795189 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.795180    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a55b9c60-64aa-4a57-82d7-6309caa50034-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.795321 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.795185    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a55b9c60-64aa-4a57-82d7-6309caa50034-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.795321 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.795215    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a55b9c60-64aa-4a57-82d7-6309caa50034-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.795321 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.795262    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a55b9c60-64aa-4a57-82d7-6309caa50034-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.795471 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.795444    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a55b9c60-64aa-4a57-82d7-6309caa50034-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.795526 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.795509    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a55b9c60-64aa-4a57-82d7-6309caa50034-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.795666 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.795644    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a55b9c60-64aa-4a57-82d7-6309caa50034-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.795836 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.795818    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a55b9c60-64aa-4a57-82d7-6309caa50034-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.797185 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.797156    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a55b9c60-64aa-4a57-82d7-6309caa50034-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.797911 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.797894    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a55b9c60-64aa-4a57-82d7-6309caa50034-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.802484 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.802456    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a55b9c60-64aa-4a57-82d7-6309caa50034-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.802612 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.802595    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhh6q\" (UniqueName: \"kubernetes.io/projected/a55b9c60-64aa-4a57-82d7-6309caa50034-kube-api-access-bhh6q\") pod \"maas-default-gateway-openshift-default-5ccb968c66-xfw54\" (UID: \"a55b9c60-64aa-4a57-82d7-6309caa50034\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:23.906548 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:23.906458    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:24.069368 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:24.069344    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"]
Jul 28 22:12:24.071989 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:12:24.071962    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda55b9c60_64aa_4a57_82d7_6309caa50034.slice/crio-024ffab9873a564b26cf0cecb1a7674a43c26117df8092f766ea2c234edd3c56 WatchSource:0}: Error finding container 024ffab9873a564b26cf0cecb1a7674a43c26117df8092f766ea2c234edd3c56: Status 404 returned error can't find the container with id 024ffab9873a564b26cf0cecb1a7674a43c26117df8092f766ea2c234edd3c56
Jul 28 22:12:24.074102 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:24.074056    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:12:24.074200 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:24.074153    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:12:24.074252 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:24.074219    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:12:24.397924 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:24.397888    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54" event={"ID":"a55b9c60-64aa-4a57-82d7-6309caa50034","Type":"ContainerStarted","Data":"d2d8553f2f199414a049d1d6083401ce8d200145a2e96774a774f0aa06e2f60d"}
Jul 28 22:12:24.398111 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:24.397950    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54" event={"ID":"a55b9c60-64aa-4a57-82d7-6309caa50034","Type":"ContainerStarted","Data":"024ffab9873a564b26cf0cecb1a7674a43c26117df8092f766ea2c234edd3c56"}
Jul 28 22:12:24.421941 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:24.421886    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54" podStartSLOduration=1.421866536 podStartE2EDuration="1.421866536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:12:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:12:24.420380723 +0000 UTC m=+473.404523342" watchObservedRunningTime="2026-07-28 22:12:24.421866536 +0000 UTC m=+473.406009158"
Jul 28 22:12:24.906790 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:24.906755    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:25.912927 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:25.912898    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:26.407566 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:26.407534    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:26.408481 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:26.408456    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:12:37.943441 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:37.943407    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-vz98t"]
Jul 28 22:12:37.946822 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:37.946805    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vz98t"
Jul 28 22:12:37.948777 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:37.948753    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-75hvs\""
Jul 28 22:12:37.953708 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:37.953682    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-vz98t"]
Jul 28 22:12:38.023907 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:38.023869    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx8wm\" (UniqueName: \"kubernetes.io/projected/eb6de888-5827-4d34-8a17-a2dd181e9bd0-kube-api-access-qx8wm\") pod \"authorino-58cd955675-vz98t\" (UID: \"eb6de888-5827-4d34-8a17-a2dd181e9bd0\") " pod="kuadrant-system/authorino-58cd955675-vz98t"
Jul 28 22:12:38.125294 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:38.125260    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx8wm\" (UniqueName: \"kubernetes.io/projected/eb6de888-5827-4d34-8a17-a2dd181e9bd0-kube-api-access-qx8wm\") pod \"authorino-58cd955675-vz98t\" (UID: \"eb6de888-5827-4d34-8a17-a2dd181e9bd0\") " pod="kuadrant-system/authorino-58cd955675-vz98t"
Jul 28 22:12:38.134218 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:38.134185    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx8wm\" (UniqueName: \"kubernetes.io/projected/eb6de888-5827-4d34-8a17-a2dd181e9bd0-kube-api-access-qx8wm\") pod \"authorino-58cd955675-vz98t\" (UID: \"eb6de888-5827-4d34-8a17-a2dd181e9bd0\") " pod="kuadrant-system/authorino-58cd955675-vz98t"
Jul 28 22:12:38.257417 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:38.257325    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vz98t"
Jul 28 22:12:38.391704 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:38.391670    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-vz98t"]
Jul 28 22:12:38.395794 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:12:38.395767    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb6de888_5827_4d34_8a17_a2dd181e9bd0.slice/crio-4275fe896265255a1044571cf27b6301e8f4362f5291438560fc81fe29489fc2 WatchSource:0}: Error finding container 4275fe896265255a1044571cf27b6301e8f4362f5291438560fc81fe29489fc2: Status 404 returned error can't find the container with id 4275fe896265255a1044571cf27b6301e8f4362f5291438560fc81fe29489fc2
Jul 28 22:12:38.460811 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:38.460775    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vz98t" event={"ID":"eb6de888-5827-4d34-8a17-a2dd181e9bd0","Type":"ContainerStarted","Data":"4275fe896265255a1044571cf27b6301e8f4362f5291438560fc81fe29489fc2"}
Jul 28 22:12:42.482252 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:42.482195    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vz98t" event={"ID":"eb6de888-5827-4d34-8a17-a2dd181e9bd0","Type":"ContainerStarted","Data":"42d27776ddd62c07bcf3248711999804a0e35a765bb91846a324d84f2dea3dbe"}
Jul 28 22:12:42.505852 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:42.505802    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-vz98t" podStartSLOduration=2.45001597 podStartE2EDuration="5.505786942s" totalImagesPullingTime="3.055770972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:12:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:12:42.505482691 +0000 UTC m=+491.489625308" watchObservedRunningTime="2026-07-28 22:12:42.505786942 +0000 UTC m=+491.489929561"
Jul 28 22:12:42.964775 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:42.964738    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-vz98t"]
Jul 28 22:12:44.491113 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:44.491070    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-vz98t" podUID="eb6de888-5827-4d34-8a17-a2dd181e9bd0" containerName="authorino" containerID="cri-o://42d27776ddd62c07bcf3248711999804a0e35a765bb91846a324d84f2dea3dbe" gracePeriod=30
Jul 28 22:12:44.737572 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:44.737548    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vz98t"
Jul 28 22:12:44.891251 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:44.891138    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qx8wm\" (UniqueName: \"kubernetes.io/projected/eb6de888-5827-4d34-8a17-a2dd181e9bd0-kube-api-access-qx8wm\") pod \"eb6de888-5827-4d34-8a17-a2dd181e9bd0\" (UID: \"eb6de888-5827-4d34-8a17-a2dd181e9bd0\") "
Jul 28 22:12:44.893222 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:44.893196    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb6de888-5827-4d34-8a17-a2dd181e9bd0-kube-api-access-qx8wm" (OuterVolumeSpecName: "kube-api-access-qx8wm") pod "eb6de888-5827-4d34-8a17-a2dd181e9bd0" (UID: "eb6de888-5827-4d34-8a17-a2dd181e9bd0"). InnerVolumeSpecName "kube-api-access-qx8wm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:12:44.992616 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:44.992577    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qx8wm\" (UniqueName: \"kubernetes.io/projected/eb6de888-5827-4d34-8a17-a2dd181e9bd0-kube-api-access-qx8wm\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:12:45.498313 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:45.498280    2574 generic.go:356] "Generic (PLEG): container finished" podID="eb6de888-5827-4d34-8a17-a2dd181e9bd0" containerID="42d27776ddd62c07bcf3248711999804a0e35a765bb91846a324d84f2dea3dbe" exitCode=0
Jul 28 22:12:45.498745 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:45.498333    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vz98t" event={"ID":"eb6de888-5827-4d34-8a17-a2dd181e9bd0","Type":"ContainerDied","Data":"42d27776ddd62c07bcf3248711999804a0e35a765bb91846a324d84f2dea3dbe"}
Jul 28 22:12:45.498745 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:45.498347    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vz98t"
Jul 28 22:12:45.498745 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:45.498367    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vz98t" event={"ID":"eb6de888-5827-4d34-8a17-a2dd181e9bd0","Type":"ContainerDied","Data":"4275fe896265255a1044571cf27b6301e8f4362f5291438560fc81fe29489fc2"}
Jul 28 22:12:45.498745 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:45.498384    2574 scope.go:117] "RemoveContainer" containerID="42d27776ddd62c07bcf3248711999804a0e35a765bb91846a324d84f2dea3dbe"
Jul 28 22:12:45.506992 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:45.506974    2574 scope.go:117] "RemoveContainer" containerID="42d27776ddd62c07bcf3248711999804a0e35a765bb91846a324d84f2dea3dbe"
Jul 28 22:12:45.507344 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:12:45.507324    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42d27776ddd62c07bcf3248711999804a0e35a765bb91846a324d84f2dea3dbe\": container with ID starting with 42d27776ddd62c07bcf3248711999804a0e35a765bb91846a324d84f2dea3dbe not found: ID does not exist" containerID="42d27776ddd62c07bcf3248711999804a0e35a765bb91846a324d84f2dea3dbe"
Jul 28 22:12:45.507426 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:45.507349    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42d27776ddd62c07bcf3248711999804a0e35a765bb91846a324d84f2dea3dbe"} err="failed to get container status \"42d27776ddd62c07bcf3248711999804a0e35a765bb91846a324d84f2dea3dbe\": rpc error: code = NotFound desc = could not find container \"42d27776ddd62c07bcf3248711999804a0e35a765bb91846a324d84f2dea3dbe\": container with ID starting with 42d27776ddd62c07bcf3248711999804a0e35a765bb91846a324d84f2dea3dbe not found: ID does not exist"
Jul 28 22:12:45.518042 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:45.518019    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-vz98t"]
Jul 28 22:12:45.521253 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:45.521213    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-vz98t"]
Jul 28 22:12:47.493585 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:12:47.493542    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb6de888-5827-4d34-8a17-a2dd181e9bd0" path="/var/lib/kubelet/pods/eb6de888-5827-4d34-8a17-a2dd181e9bd0/volumes"
Jul 28 22:13:16.006138 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.006105    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7b9b4d9c46-mzzcf"]
Jul 28 22:13:16.006627 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.006464    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb6de888-5827-4d34-8a17-a2dd181e9bd0" containerName="authorino"
Jul 28 22:13:16.006627 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.006475    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb6de888-5827-4d34-8a17-a2dd181e9bd0" containerName="authorino"
Jul 28 22:13:16.006627 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.006545    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb6de888-5827-4d34-8a17-a2dd181e9bd0" containerName="authorino"
Jul 28 22:13:16.014842 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.011722    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b9b4d9c46-mzzcf"
Jul 28 22:13:16.028163 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.027951    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7v8tm\""
Jul 28 22:13:16.028348 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.028219    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 22:13:16.030087 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.028527    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 22:13:16.037215 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.037186    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b9b4d9c46-mzzcf"]
Jul 28 22:13:16.051049 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.051017    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a5f5602-1c7e-49c5-9916-1675b9981c72-webhook-cert\") pod \"maas-controller-7b9b4d9c46-mzzcf\" (UID: \"4a5f5602-1c7e-49c5-9916-1675b9981c72\") " pod="opendatahub/maas-controller-7b9b4d9c46-mzzcf"
Jul 28 22:13:16.051255 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.051073    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-586c7\" (UniqueName: \"kubernetes.io/projected/4a5f5602-1c7e-49c5-9916-1675b9981c72-kube-api-access-586c7\") pod \"maas-controller-7b9b4d9c46-mzzcf\" (UID: \"4a5f5602-1c7e-49c5-9916-1675b9981c72\") " pod="opendatahub/maas-controller-7b9b4d9c46-mzzcf"
Jul 28 22:13:16.152350 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.152312    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a5f5602-1c7e-49c5-9916-1675b9981c72-webhook-cert\") pod \"maas-controller-7b9b4d9c46-mzzcf\" (UID: \"4a5f5602-1c7e-49c5-9916-1675b9981c72\") " pod="opendatahub/maas-controller-7b9b4d9c46-mzzcf"
Jul 28 22:13:16.152544 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.152355    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-586c7\" (UniqueName: \"kubernetes.io/projected/4a5f5602-1c7e-49c5-9916-1675b9981c72-kube-api-access-586c7\") pod \"maas-controller-7b9b4d9c46-mzzcf\" (UID: \"4a5f5602-1c7e-49c5-9916-1675b9981c72\") " pod="opendatahub/maas-controller-7b9b4d9c46-mzzcf"
Jul 28 22:13:16.154878 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.154845    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a5f5602-1c7e-49c5-9916-1675b9981c72-webhook-cert\") pod \"maas-controller-7b9b4d9c46-mzzcf\" (UID: \"4a5f5602-1c7e-49c5-9916-1675b9981c72\") " pod="opendatahub/maas-controller-7b9b4d9c46-mzzcf"
Jul 28 22:13:16.160877 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.160847    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-586c7\" (UniqueName: \"kubernetes.io/projected/4a5f5602-1c7e-49c5-9916-1675b9981c72-kube-api-access-586c7\") pod \"maas-controller-7b9b4d9c46-mzzcf\" (UID: \"4a5f5602-1c7e-49c5-9916-1675b9981c72\") " pod="opendatahub/maas-controller-7b9b4d9c46-mzzcf"
Jul 28 22:13:16.179353 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.179318    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-796f547645-mf4pz"]
Jul 28 22:13:16.181689 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.181667    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-796f547645-mf4pz"
Jul 28 22:13:16.191257 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.191213    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-796f547645-mf4pz"]
Jul 28 22:13:16.253784 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.253749    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4zsn\" (UniqueName: \"kubernetes.io/projected/cf558383-1c95-4f71-9ada-1c9e165f2ecb-kube-api-access-x4zsn\") pod \"maas-controller-796f547645-mf4pz\" (UID: \"cf558383-1c95-4f71-9ada-1c9e165f2ecb\") " pod="opendatahub/maas-controller-796f547645-mf4pz"
Jul 28 22:13:16.253971 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.253816    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf558383-1c95-4f71-9ada-1c9e165f2ecb-webhook-cert\") pod \"maas-controller-796f547645-mf4pz\" (UID: \"cf558383-1c95-4f71-9ada-1c9e165f2ecb\") " pod="opendatahub/maas-controller-796f547645-mf4pz"
Jul 28 22:13:16.345044 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.344943    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b9b4d9c46-mzzcf"
Jul 28 22:13:16.354357 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.354334    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf558383-1c95-4f71-9ada-1c9e165f2ecb-webhook-cert\") pod \"maas-controller-796f547645-mf4pz\" (UID: \"cf558383-1c95-4f71-9ada-1c9e165f2ecb\") " pod="opendatahub/maas-controller-796f547645-mf4pz"
Jul 28 22:13:16.354430 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.354412    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4zsn\" (UniqueName: \"kubernetes.io/projected/cf558383-1c95-4f71-9ada-1c9e165f2ecb-kube-api-access-x4zsn\") pod \"maas-controller-796f547645-mf4pz\" (UID: \"cf558383-1c95-4f71-9ada-1c9e165f2ecb\") " pod="opendatahub/maas-controller-796f547645-mf4pz"
Jul 28 22:13:16.356669 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.356638    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf558383-1c95-4f71-9ada-1c9e165f2ecb-webhook-cert\") pod \"maas-controller-796f547645-mf4pz\" (UID: \"cf558383-1c95-4f71-9ada-1c9e165f2ecb\") " pod="opendatahub/maas-controller-796f547645-mf4pz"
Jul 28 22:13:16.362928 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.362898    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4zsn\" (UniqueName: \"kubernetes.io/projected/cf558383-1c95-4f71-9ada-1c9e165f2ecb-kube-api-access-x4zsn\") pod \"maas-controller-796f547645-mf4pz\" (UID: \"cf558383-1c95-4f71-9ada-1c9e165f2ecb\") " pod="opendatahub/maas-controller-796f547645-mf4pz"
Jul 28 22:13:16.480901 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.480867    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b9b4d9c46-mzzcf"]
Jul 28 22:13:16.484555 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:13:16.484521    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a5f5602_1c7e_49c5_9916_1675b9981c72.slice/crio-f172bace2f582f8225cb27bab7de3be246d6aaa2046208e7591e5c88b25919b2 WatchSource:0}: Error finding container f172bace2f582f8225cb27bab7de3be246d6aaa2046208e7591e5c88b25919b2: Status 404 returned error can't find the container with id f172bace2f582f8225cb27bab7de3be246d6aaa2046208e7591e5c88b25919b2
Jul 28 22:13:16.492760 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.492734    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-796f547645-mf4pz"
Jul 28 22:13:16.628069 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.627987    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b9b4d9c46-mzzcf" event={"ID":"4a5f5602-1c7e-49c5-9916-1675b9981c72","Type":"ContainerStarted","Data":"f172bace2f582f8225cb27bab7de3be246d6aaa2046208e7591e5c88b25919b2"}
Jul 28 22:13:16.641736 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:16.641702    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-796f547645-mf4pz"]
Jul 28 22:13:16.644654 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:13:16.644624    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf558383_1c95_4f71_9ada_1c9e165f2ecb.slice/crio-f33eafdfc76ec0de4c9568187bc1e7041e2b91e2f06bf394c42d3ebd716350ae WatchSource:0}: Error finding container f33eafdfc76ec0de4c9568187bc1e7041e2b91e2f06bf394c42d3ebd716350ae: Status 404 returned error can't find the container with id f33eafdfc76ec0de4c9568187bc1e7041e2b91e2f06bf394c42d3ebd716350ae
Jul 28 22:13:17.636840 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:17.636783    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-796f547645-mf4pz" event={"ID":"cf558383-1c95-4f71-9ada-1c9e165f2ecb","Type":"ContainerStarted","Data":"f33eafdfc76ec0de4c9568187bc1e7041e2b91e2f06bf394c42d3ebd716350ae"}
Jul 28 22:13:19.649901 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:19.649854    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b9b4d9c46-mzzcf" event={"ID":"4a5f5602-1c7e-49c5-9916-1675b9981c72","Type":"ContainerStarted","Data":"e59bc303002dce519e80124832c1a9117f74ff2ce40108dd3486cc29ff2afdf7"}
Jul 28 22:13:19.650428 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:19.650026    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7b9b4d9c46-mzzcf"
Jul 28 22:13:19.651665 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:19.651641    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-796f547645-mf4pz" event={"ID":"cf558383-1c95-4f71-9ada-1c9e165f2ecb","Type":"ContainerStarted","Data":"00d375b4e2eae92c96b751abee59b78d7a86893eec9676952b1dedfc421378ba"}
Jul 28 22:13:19.651806 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:19.651746    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-796f547645-mf4pz"
Jul 28 22:13:19.670676 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:19.670620    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7b9b4d9c46-mzzcf" podStartSLOduration=2.217980034 podStartE2EDuration="4.670596458s" totalImagesPullingTime="2.452616424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:13:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:13:19.668304765 +0000 UTC m=+528.652447385" watchObservedRunningTime="2026-07-28 22:13:19.670596458 +0000 UTC m=+528.654739079"
Jul 28 22:13:19.685362 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:19.685305    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-796f547645-mf4pz" podStartSLOduration=1.402282872 podStartE2EDuration="3.685288662s" totalImagesPullingTime="2.28300579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:13:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:13:19.684716239 +0000 UTC m=+528.668858871" watchObservedRunningTime="2026-07-28 22:13:19.685288662 +0000 UTC m=+528.669431282"
Jul 28 22:13:30.659523 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:30.659446    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7b9b4d9c46-mzzcf"
Jul 28 22:13:30.659947 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:30.659623    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-796f547645-mf4pz"
Jul 28 22:13:30.724682 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:30.724649    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b9b4d9c46-mzzcf"]
Jul 28 22:13:30.724875 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:30.724836    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7b9b4d9c46-mzzcf" podUID="4a5f5602-1c7e-49c5-9916-1675b9981c72" containerName="manager" containerID="cri-o://e59bc303002dce519e80124832c1a9117f74ff2ce40108dd3486cc29ff2afdf7" gracePeriod=30
Jul 28 22:13:30.969913 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:30.969891    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b9b4d9c46-mzzcf"
Jul 28 22:13:31.087712 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:31.087679    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a5f5602-1c7e-49c5-9916-1675b9981c72-webhook-cert\") pod \"4a5f5602-1c7e-49c5-9916-1675b9981c72\" (UID: \"4a5f5602-1c7e-49c5-9916-1675b9981c72\") "
Jul 28 22:13:31.087884 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:31.087789    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-586c7\" (UniqueName: \"kubernetes.io/projected/4a5f5602-1c7e-49c5-9916-1675b9981c72-kube-api-access-586c7\") pod \"4a5f5602-1c7e-49c5-9916-1675b9981c72\" (UID: \"4a5f5602-1c7e-49c5-9916-1675b9981c72\") "
Jul 28 22:13:31.089884 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:31.089851    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a5f5602-1c7e-49c5-9916-1675b9981c72-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "4a5f5602-1c7e-49c5-9916-1675b9981c72" (UID: "4a5f5602-1c7e-49c5-9916-1675b9981c72"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:13:31.089998 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:31.089911    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a5f5602-1c7e-49c5-9916-1675b9981c72-kube-api-access-586c7" (OuterVolumeSpecName: "kube-api-access-586c7") pod "4a5f5602-1c7e-49c5-9916-1675b9981c72" (UID: "4a5f5602-1c7e-49c5-9916-1675b9981c72"). InnerVolumeSpecName "kube-api-access-586c7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:13:31.189306 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:31.189266    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-586c7\" (UniqueName: \"kubernetes.io/projected/4a5f5602-1c7e-49c5-9916-1675b9981c72-kube-api-access-586c7\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:13:31.189306 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:31.189298    2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a5f5602-1c7e-49c5-9916-1675b9981c72-webhook-cert\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:13:31.710011 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:31.709977    2574 generic.go:356] "Generic (PLEG): container finished" podID="4a5f5602-1c7e-49c5-9916-1675b9981c72" containerID="e59bc303002dce519e80124832c1a9117f74ff2ce40108dd3486cc29ff2afdf7" exitCode=0
Jul 28 22:13:31.710507 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:31.710019    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b9b4d9c46-mzzcf" event={"ID":"4a5f5602-1c7e-49c5-9916-1675b9981c72","Type":"ContainerDied","Data":"e59bc303002dce519e80124832c1a9117f74ff2ce40108dd3486cc29ff2afdf7"}
Jul 28 22:13:31.710507 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:31.710025    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b9b4d9c46-mzzcf"
Jul 28 22:13:31.710507 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:31.710063    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b9b4d9c46-mzzcf" event={"ID":"4a5f5602-1c7e-49c5-9916-1675b9981c72","Type":"ContainerDied","Data":"f172bace2f582f8225cb27bab7de3be246d6aaa2046208e7591e5c88b25919b2"}
Jul 28 22:13:31.710507 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:31.710086    2574 scope.go:117] "RemoveContainer" containerID="e59bc303002dce519e80124832c1a9117f74ff2ce40108dd3486cc29ff2afdf7"
Jul 28 22:13:31.718632 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:31.718611    2574 scope.go:117] "RemoveContainer" containerID="e59bc303002dce519e80124832c1a9117f74ff2ce40108dd3486cc29ff2afdf7"
Jul 28 22:13:31.718925 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:13:31.718905    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e59bc303002dce519e80124832c1a9117f74ff2ce40108dd3486cc29ff2afdf7\": container with ID starting with e59bc303002dce519e80124832c1a9117f74ff2ce40108dd3486cc29ff2afdf7 not found: ID does not exist" containerID="e59bc303002dce519e80124832c1a9117f74ff2ce40108dd3486cc29ff2afdf7"
Jul 28 22:13:31.718981 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:31.718934    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e59bc303002dce519e80124832c1a9117f74ff2ce40108dd3486cc29ff2afdf7"} err="failed to get container status \"e59bc303002dce519e80124832c1a9117f74ff2ce40108dd3486cc29ff2afdf7\": rpc error: code = NotFound desc = could not find container \"e59bc303002dce519e80124832c1a9117f74ff2ce40108dd3486cc29ff2afdf7\": container with ID starting with e59bc303002dce519e80124832c1a9117f74ff2ce40108dd3486cc29ff2afdf7 not found: ID does not exist"
Jul 28 22:13:31.727298 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:31.727271    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b9b4d9c46-mzzcf"]
Jul 28 22:13:31.730197 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:31.730173    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7b9b4d9c46-mzzcf"]
Jul 28 22:13:33.493835 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:33.493802    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a5f5602-1c7e-49c5-9916-1675b9981c72" path="/var/lib/kubelet/pods/4a5f5602-1c7e-49c5-9916-1675b9981c72/volumes"
Jul 28 22:13:51.557071 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:51.557033    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8dd88f89-pj89b"]
Jul 28 22:13:51.557604 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:51.557578    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a5f5602-1c7e-49c5-9916-1675b9981c72" containerName="manager"
Jul 28 22:13:51.557604 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:51.557596    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a5f5602-1c7e-49c5-9916-1675b9981c72" containerName="manager"
Jul 28 22:13:51.557712 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:51.557682    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a5f5602-1c7e-49c5-9916-1675b9981c72" containerName="manager"
Jul 28 22:13:51.562774 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:51.562750    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8dd88f89-pj89b"
Jul 28 22:13:51.564914 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:51.564885    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 28 22:13:51.565066 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:51.565049    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 22:13:51.565396 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:51.565376    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 22:13:51.565535 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:51.565377    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tlhn2\""
Jul 28 22:13:51.571852 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:51.571766    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8dd88f89-pj89b"]
Jul 28 22:13:51.672889 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:51.672855    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c814da8-6e2d-476f-b0b9-d3d2eaaa02b8-maas-api-tls\") pod \"maas-api-8dd88f89-pj89b\" (UID: \"6c814da8-6e2d-476f-b0b9-d3d2eaaa02b8\") " pod="odh-ai-gateway-infra/maas-api-8dd88f89-pj89b"
Jul 28 22:13:51.672889 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:51.672889    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c814da8-6e2d-476f-b0b9-d3d2eaaa02b8-openshift-service-ca\") pod \"maas-api-8dd88f89-pj89b\" (UID: \"6c814da8-6e2d-476f-b0b9-d3d2eaaa02b8\") " pod="odh-ai-gateway-infra/maas-api-8dd88f89-pj89b"
Jul 28 22:13:51.672889 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:51.672908    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9qsx\" (UniqueName: \"kubernetes.io/projected/6c814da8-6e2d-476f-b0b9-d3d2eaaa02b8-kube-api-access-c9qsx\") pod \"maas-api-8dd88f89-pj89b\" (UID: \"6c814da8-6e2d-476f-b0b9-d3d2eaaa02b8\") " pod="odh-ai-gateway-infra/maas-api-8dd88f89-pj89b"
Jul 28 22:13:51.774190 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:51.774157    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c814da8-6e2d-476f-b0b9-d3d2eaaa02b8-maas-api-tls\") pod \"maas-api-8dd88f89-pj89b\" (UID: \"6c814da8-6e2d-476f-b0b9-d3d2eaaa02b8\") " pod="odh-ai-gateway-infra/maas-api-8dd88f89-pj89b"
Jul 28 22:13:51.774190 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:51.774189    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c814da8-6e2d-476f-b0b9-d3d2eaaa02b8-openshift-service-ca\") pod \"maas-api-8dd88f89-pj89b\" (UID: \"6c814da8-6e2d-476f-b0b9-d3d2eaaa02b8\") " pod="odh-ai-gateway-infra/maas-api-8dd88f89-pj89b"
Jul 28 22:13:51.774431 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:51.774206    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9qsx\" (UniqueName: \"kubernetes.io/projected/6c814da8-6e2d-476f-b0b9-d3d2eaaa02b8-kube-api-access-c9qsx\") pod \"maas-api-8dd88f89-pj89b\" (UID: \"6c814da8-6e2d-476f-b0b9-d3d2eaaa02b8\") " pod="odh-ai-gateway-infra/maas-api-8dd88f89-pj89b"
Jul 28 22:13:51.774955 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:51.774937    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c814da8-6e2d-476f-b0b9-d3d2eaaa02b8-openshift-service-ca\") pod \"maas-api-8dd88f89-pj89b\" (UID: \"6c814da8-6e2d-476f-b0b9-d3d2eaaa02b8\") " pod="odh-ai-gateway-infra/maas-api-8dd88f89-pj89b"
Jul 28 22:13:51.776565 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:51.776546    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c814da8-6e2d-476f-b0b9-d3d2eaaa02b8-maas-api-tls\") pod \"maas-api-8dd88f89-pj89b\" (UID: \"6c814da8-6e2d-476f-b0b9-d3d2eaaa02b8\") " pod="odh-ai-gateway-infra/maas-api-8dd88f89-pj89b"
Jul 28 22:13:51.783425 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:51.783395    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9qsx\" (UniqueName: \"kubernetes.io/projected/6c814da8-6e2d-476f-b0b9-d3d2eaaa02b8-kube-api-access-c9qsx\") pod \"maas-api-8dd88f89-pj89b\" (UID: \"6c814da8-6e2d-476f-b0b9-d3d2eaaa02b8\") " pod="odh-ai-gateway-infra/maas-api-8dd88f89-pj89b"
Jul 28 22:13:51.875814 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:51.875741    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8dd88f89-pj89b"
Jul 28 22:13:52.020850 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:52.020819    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8dd88f89-pj89b"]
Jul 28 22:13:52.024491 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:13:52.024454    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c814da8_6e2d_476f_b0b9_d3d2eaaa02b8.slice/crio-564e8bd79d7bde58e28165d9302a24fb4b02dd66b7923729d49d141cd54905e1 WatchSource:0}: Error finding container 564e8bd79d7bde58e28165d9302a24fb4b02dd66b7923729d49d141cd54905e1: Status 404 returned error can't find the container with id 564e8bd79d7bde58e28165d9302a24fb4b02dd66b7923729d49d141cd54905e1
Jul 28 22:13:52.242523 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:52.239957    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-m6qn9"]
Jul 28 22:13:52.244608 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:52.244582    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m6qn9"
Jul 28 22:13:52.246965 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:52.246941    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 28 22:13:52.247794 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:52.247749    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-8ssmp\""
Jul 28 22:13:52.253323 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:52.253295    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-m6qn9"]
Jul 28 22:13:52.380129 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:52.380098    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1afb94a8-a577-4cab-93f7-e3cab182cd83-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-m6qn9\" (UID: \"1afb94a8-a577-4cab-93f7-e3cab182cd83\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m6qn9"
Jul 28 22:13:52.380129 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:52.380135    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpz9r\" (UniqueName: \"kubernetes.io/projected/1afb94a8-a577-4cab-93f7-e3cab182cd83-kube-api-access-zpz9r\") pod \"payload-pre-processing-687dbb6cf6-m6qn9\" (UID: \"1afb94a8-a577-4cab-93f7-e3cab182cd83\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m6qn9"
Jul 28 22:13:52.481086 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:52.481044    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpz9r\" (UniqueName: \"kubernetes.io/projected/1afb94a8-a577-4cab-93f7-e3cab182cd83-kube-api-access-zpz9r\") pod \"payload-pre-processing-687dbb6cf6-m6qn9\" (UID: \"1afb94a8-a577-4cab-93f7-e3cab182cd83\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m6qn9"
Jul 28 22:13:52.481318 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:52.481149    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1afb94a8-a577-4cab-93f7-e3cab182cd83-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-m6qn9\" (UID: \"1afb94a8-a577-4cab-93f7-e3cab182cd83\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m6qn9"
Jul 28 22:13:52.481808 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:52.481789    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1afb94a8-a577-4cab-93f7-e3cab182cd83-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-m6qn9\" (UID: \"1afb94a8-a577-4cab-93f7-e3cab182cd83\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m6qn9"
Jul 28 22:13:52.490228 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:52.490200    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpz9r\" (UniqueName: \"kubernetes.io/projected/1afb94a8-a577-4cab-93f7-e3cab182cd83-kube-api-access-zpz9r\") pod \"payload-pre-processing-687dbb6cf6-m6qn9\" (UID: \"1afb94a8-a577-4cab-93f7-e3cab182cd83\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m6qn9"
Jul 28 22:13:52.557569 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:52.557487    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m6qn9"
Jul 28 22:13:52.721061 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:52.720984    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-m6qn9"]
Jul 28 22:13:52.723790 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:13:52.723757    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1afb94a8_a577_4cab_93f7_e3cab182cd83.slice/crio-4d36d18d737262e9d6003758112d6c5528bee447d39fe080213533d0135b1257 WatchSource:0}: Error finding container 4d36d18d737262e9d6003758112d6c5528bee447d39fe080213533d0135b1257: Status 404 returned error can't find the container with id 4d36d18d737262e9d6003758112d6c5528bee447d39fe080213533d0135b1257
Jul 28 22:13:52.804876 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:52.804838    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8dd88f89-pj89b" event={"ID":"6c814da8-6e2d-476f-b0b9-d3d2eaaa02b8","Type":"ContainerStarted","Data":"564e8bd79d7bde58e28165d9302a24fb4b02dd66b7923729d49d141cd54905e1"}
Jul 28 22:13:52.806298 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:52.806263    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m6qn9" event={"ID":"1afb94a8-a577-4cab-93f7-e3cab182cd83","Type":"ContainerStarted","Data":"4d36d18d737262e9d6003758112d6c5528bee447d39fe080213533d0135b1257"}
Jul 28 22:13:54.819438 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:54.819402    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8dd88f89-pj89b" event={"ID":"6c814da8-6e2d-476f-b0b9-d3d2eaaa02b8","Type":"ContainerStarted","Data":"55163fe3d883afbf26f1a771aba71ab991da1530e0d3fbb87be999e8def8c0a1"}
Jul 28 22:13:54.819965 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:54.819520    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8dd88f89-pj89b"
Jul 28 22:13:55.825207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:55.825167    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m6qn9" event={"ID":"1afb94a8-a577-4cab-93f7-e3cab182cd83","Type":"ContainerStarted","Data":"ab15b57c6a78644400b42fa517a2758556959aac6159bbd03e53461cc86b5189"}
Jul 28 22:13:55.825620 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:55.825359    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m6qn9"
Jul 28 22:13:55.843190 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:55.841636    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8dd88f89-pj89b" podStartSLOduration=2.936882235 podStartE2EDuration="4.841614863s" totalImagesPullingTime="1.904732628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:13:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:13:54.853500998 +0000 UTC m=+563.837643619" watchObservedRunningTime="2026-07-28 22:13:55.841614863 +0000 UTC m=+564.825757484"
Jul 28 22:13:55.843190 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:55.842459    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m6qn9" podStartSLOduration=0.945609589 podStartE2EDuration="3.84244854s" totalImagesPullingTime="2.896838951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:13:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:13:55.839568064 +0000 UTC m=+564.823710685" watchObservedRunningTime="2026-07-28 22:13:55.84244854 +0000 UTC m=+564.826591161"
Jul 28 22:13:59.845816 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:59.845779    2574 generic.go:356] "Generic (PLEG): container finished" podID="a55b9c60-64aa-4a57-82d7-6309caa50034" containerID="d2d8553f2f199414a049d1d6083401ce8d200145a2e96774a774f0aa06e2f60d" exitCode=0
Jul 28 22:13:59.846271 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:59.845851    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54" event={"ID":"a55b9c60-64aa-4a57-82d7-6309caa50034","Type":"ContainerDied","Data":"d2d8553f2f199414a049d1d6083401ce8d200145a2e96774a774f0aa06e2f60d"}
Jul 28 22:13:59.846271 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:59.846225    2574 scope.go:117] "RemoveContainer" containerID="d2d8553f2f199414a049d1d6083401ce8d200145a2e96774a774f0aa06e2f60d"
Jul 28 22:13:59.847161 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:59.847137    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:13:59.847218 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:59.847206    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:13:59.847286 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:13:59.847255    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:14:00.831119 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:00.831091    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8dd88f89-pj89b"
Jul 28 22:14:00.858989 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:00.858945    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54" event={"ID":"a55b9c60-64aa-4a57-82d7-6309caa50034","Type":"ContainerStarted","Data":"08414cf550a302bf2e527275d90d19e1a9197233144849eadaaf789c52f9d56a"}
Jul 28 22:14:00.907052 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:00.907024    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:14:00.907345 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:00.907321    2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-xfw54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Jul 28 22:14:00.907415 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:00.907391    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54" podUID="a55b9c60-64aa-4a57-82d7-6309caa50034" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Jul 28 22:14:01.907100 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:01.907066    2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-xfw54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Jul 28 22:14:01.907955 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:01.907121    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54" podUID="a55b9c60-64aa-4a57-82d7-6309caa50034" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Jul 28 22:14:02.907038 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:02.906994    2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-xfw54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Jul 28 22:14:02.907263 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:02.907072    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54" podUID="a55b9c60-64aa-4a57-82d7-6309caa50034" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Jul 28 22:14:03.907315 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:03.907281    2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-xfw54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Jul 28 22:14:03.907714 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:03.907334    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54" podUID="a55b9c60-64aa-4a57-82d7-6309caa50034" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Jul 28 22:14:04.951028 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:04.950991    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:14:04.951491 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:04.951273    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:14:04.952176 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:04.952155    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-xfw54"
Jul 28 22:14:06.831414 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:06.831382    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m6qn9"
Jul 28 22:14:32.007430 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.007394    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"]
Jul 28 22:14:32.013848 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.013815    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:32.016589 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.016565    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 22:14:32.016589 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.016587    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 28 22:14:32.016871 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.016854    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2wf7s\""
Jul 28 22:14:32.017003 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.016899    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 22:14:32.019543 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.019522    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"]
Jul 28 22:14:32.118093 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.118057    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05fb5c53-d53e-42fd-b037-966389f058e6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvb8r\" (UID: \"05fb5c53-d53e-42fd-b037-966389f058e6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:32.118298 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.118104    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05fb5c53-d53e-42fd-b037-966389f058e6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvb8r\" (UID: \"05fb5c53-d53e-42fd-b037-966389f058e6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:32.118298 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.118135    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05fb5c53-d53e-42fd-b037-966389f058e6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvb8r\" (UID: \"05fb5c53-d53e-42fd-b037-966389f058e6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:32.118298 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.118189    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rzv2\" (UniqueName: \"kubernetes.io/projected/05fb5c53-d53e-42fd-b037-966389f058e6-kube-api-access-6rzv2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvb8r\" (UID: \"05fb5c53-d53e-42fd-b037-966389f058e6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:32.118298 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.118264    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05fb5c53-d53e-42fd-b037-966389f058e6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvb8r\" (UID: \"05fb5c53-d53e-42fd-b037-966389f058e6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:32.118443 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.118323    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05fb5c53-d53e-42fd-b037-966389f058e6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvb8r\" (UID: \"05fb5c53-d53e-42fd-b037-966389f058e6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:32.219529 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.219489    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05fb5c53-d53e-42fd-b037-966389f058e6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvb8r\" (UID: \"05fb5c53-d53e-42fd-b037-966389f058e6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:32.219715 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.219539    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05fb5c53-d53e-42fd-b037-966389f058e6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvb8r\" (UID: \"05fb5c53-d53e-42fd-b037-966389f058e6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:32.219715 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.219560    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05fb5c53-d53e-42fd-b037-966389f058e6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvb8r\" (UID: \"05fb5c53-d53e-42fd-b037-966389f058e6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:32.219715 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.219586    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05fb5c53-d53e-42fd-b037-966389f058e6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvb8r\" (UID: \"05fb5c53-d53e-42fd-b037-966389f058e6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:32.219928 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.219706    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rzv2\" (UniqueName: \"kubernetes.io/projected/05fb5c53-d53e-42fd-b037-966389f058e6-kube-api-access-6rzv2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvb8r\" (UID: \"05fb5c53-d53e-42fd-b037-966389f058e6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:32.219928 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.219779    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05fb5c53-d53e-42fd-b037-966389f058e6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvb8r\" (UID: \"05fb5c53-d53e-42fd-b037-966389f058e6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:32.220001 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.219933    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05fb5c53-d53e-42fd-b037-966389f058e6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvb8r\" (UID: \"05fb5c53-d53e-42fd-b037-966389f058e6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:32.220001 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.219963    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05fb5c53-d53e-42fd-b037-966389f058e6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvb8r\" (UID: \"05fb5c53-d53e-42fd-b037-966389f058e6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:32.220076 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.220005    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05fb5c53-d53e-42fd-b037-966389f058e6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvb8r\" (UID: \"05fb5c53-d53e-42fd-b037-966389f058e6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:32.221949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.221923    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05fb5c53-d53e-42fd-b037-966389f058e6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvb8r\" (UID: \"05fb5c53-d53e-42fd-b037-966389f058e6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:32.222160 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.222143    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05fb5c53-d53e-42fd-b037-966389f058e6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvb8r\" (UID: \"05fb5c53-d53e-42fd-b037-966389f058e6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:32.226999 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.226970    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rzv2\" (UniqueName: \"kubernetes.io/projected/05fb5c53-d53e-42fd-b037-966389f058e6-kube-api-access-6rzv2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvb8r\" (UID: \"05fb5c53-d53e-42fd-b037-966389f058e6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:32.325876 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.325786    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:32.459996 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:32.459635    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"]
Jul 28 22:14:32.462280 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:14:32.462250    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05fb5c53_d53e_42fd_b037_966389f058e6.slice/crio-c1cb08e94b0edcc98c9a6234b0c1bd6a7abda68e33a678ef7cce0ab3f0fd6ad8 WatchSource:0}: Error finding container c1cb08e94b0edcc98c9a6234b0c1bd6a7abda68e33a678ef7cce0ab3f0fd6ad8: Status 404 returned error can't find the container with id c1cb08e94b0edcc98c9a6234b0c1bd6a7abda68e33a678ef7cce0ab3f0fd6ad8
Jul 28 22:14:33.009145 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:33.009103    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r" event={"ID":"05fb5c53-d53e-42fd-b037-966389f058e6","Type":"ContainerStarted","Data":"c1cb08e94b0edcc98c9a6234b0c1bd6a7abda68e33a678ef7cce0ab3f0fd6ad8"}
Jul 28 22:14:34.015262 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:34.015211    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r" event={"ID":"05fb5c53-d53e-42fd-b037-966389f058e6","Type":"ContainerStarted","Data":"ccd272b127a7fff04df29025962c98b0971e688d3300fd56175cd91abb0ebd65"}
Jul 28 22:14:34.043642 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:34.043582    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r" podStartSLOduration=2.107713483 podStartE2EDuration="3.043561312s" totalImagesPullingTime="935.847829ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:14:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:14:34.041904065 +0000 UTC m=+603.026046686" watchObservedRunningTime="2026-07-28 22:14:34.043561312 +0000 UTC m=+603.027703933"
Jul 28 22:14:42.326314 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:42.326271    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:42.326730 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:42.326338    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:42.338825 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:42.338796    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:14:43.068201 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:14:43.068175    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvb8r"
Jul 28 22:15:05.786605 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:05.786518    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"]
Jul 28 22:15:05.790054 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:05.790032    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:05.792057 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:05.792038    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 28 22:15:05.798636 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:05.798612    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"]
Jul 28 22:15:05.823988 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:05.823955    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f2cb5ec-81c7-4bc3-98a4-99853ad7057f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn\" (UID: \"2f2cb5ec-81c7-4bc3-98a4-99853ad7057f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:05.824162 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:05.823996    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f2cb5ec-81c7-4bc3-98a4-99853ad7057f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn\" (UID: \"2f2cb5ec-81c7-4bc3-98a4-99853ad7057f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:05.824162 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:05.824022    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f2cb5ec-81c7-4bc3-98a4-99853ad7057f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn\" (UID: \"2f2cb5ec-81c7-4bc3-98a4-99853ad7057f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:05.824162 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:05.824040    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f2cb5ec-81c7-4bc3-98a4-99853ad7057f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn\" (UID: \"2f2cb5ec-81c7-4bc3-98a4-99853ad7057f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:05.824162 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:05.824091    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x6dq\" (UniqueName: \"kubernetes.io/projected/2f2cb5ec-81c7-4bc3-98a4-99853ad7057f-kube-api-access-4x6dq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn\" (UID: \"2f2cb5ec-81c7-4bc3-98a4-99853ad7057f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:05.824162 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:05.824133    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f2cb5ec-81c7-4bc3-98a4-99853ad7057f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn\" (UID: \"2f2cb5ec-81c7-4bc3-98a4-99853ad7057f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:05.925008 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:05.924971    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f2cb5ec-81c7-4bc3-98a4-99853ad7057f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn\" (UID: \"2f2cb5ec-81c7-4bc3-98a4-99853ad7057f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:05.925008 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:05.925009    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f2cb5ec-81c7-4bc3-98a4-99853ad7057f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn\" (UID: \"2f2cb5ec-81c7-4bc3-98a4-99853ad7057f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:05.925211 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:05.925041    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f2cb5ec-81c7-4bc3-98a4-99853ad7057f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn\" (UID: \"2f2cb5ec-81c7-4bc3-98a4-99853ad7057f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:05.925279 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:05.925227    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f2cb5ec-81c7-4bc3-98a4-99853ad7057f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn\" (UID: \"2f2cb5ec-81c7-4bc3-98a4-99853ad7057f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:05.925339 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:05.925294    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x6dq\" (UniqueName: \"kubernetes.io/projected/2f2cb5ec-81c7-4bc3-98a4-99853ad7057f-kube-api-access-4x6dq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn\" (UID: \"2f2cb5ec-81c7-4bc3-98a4-99853ad7057f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:05.925391 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:05.925335    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f2cb5ec-81c7-4bc3-98a4-99853ad7057f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn\" (UID: \"2f2cb5ec-81c7-4bc3-98a4-99853ad7057f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:05.925599 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:05.925568    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f2cb5ec-81c7-4bc3-98a4-99853ad7057f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn\" (UID: \"2f2cb5ec-81c7-4bc3-98a4-99853ad7057f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:05.925735 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:05.925608    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f2cb5ec-81c7-4bc3-98a4-99853ad7057f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn\" (UID: \"2f2cb5ec-81c7-4bc3-98a4-99853ad7057f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:05.925735 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:05.925680    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f2cb5ec-81c7-4bc3-98a4-99853ad7057f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn\" (UID: \"2f2cb5ec-81c7-4bc3-98a4-99853ad7057f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:05.927390 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:05.927363    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f2cb5ec-81c7-4bc3-98a4-99853ad7057f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn\" (UID: \"2f2cb5ec-81c7-4bc3-98a4-99853ad7057f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:05.927554 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:05.927537    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f2cb5ec-81c7-4bc3-98a4-99853ad7057f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn\" (UID: \"2f2cb5ec-81c7-4bc3-98a4-99853ad7057f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:05.932336 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:05.932312    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x6dq\" (UniqueName: \"kubernetes.io/projected/2f2cb5ec-81c7-4bc3-98a4-99853ad7057f-kube-api-access-4x6dq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn\" (UID: \"2f2cb5ec-81c7-4bc3-98a4-99853ad7057f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:06.100300 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:06.100174    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:06.235526 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:06.235497    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"]
Jul 28 22:15:06.238922 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:15:06.238890    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f2cb5ec_81c7_4bc3_98a4_99853ad7057f.slice/crio-614541de9161bc0116460ead1c083ea01e44dc8c5ab5109a1940291aa3abb4c1 WatchSource:0}: Error finding container 614541de9161bc0116460ead1c083ea01e44dc8c5ab5109a1940291aa3abb4c1: Status 404 returned error can't find the container with id 614541de9161bc0116460ead1c083ea01e44dc8c5ab5109a1940291aa3abb4c1
Jul 28 22:15:06.240274 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:06.240227    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 22:15:07.170796 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:07.170762    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn" event={"ID":"2f2cb5ec-81c7-4bc3-98a4-99853ad7057f","Type":"ContainerStarted","Data":"0e03a44840cfe624f141b70a8ca3e5d67d483aff62e4f3419b7dcb0705ac0bb7"}
Jul 28 22:15:07.170796 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:07.170802    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn" event={"ID":"2f2cb5ec-81c7-4bc3-98a4-99853ad7057f","Type":"ContainerStarted","Data":"614541de9161bc0116460ead1c083ea01e44dc8c5ab5109a1940291aa3abb4c1"}
Jul 28 22:15:07.200710 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:07.200661    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn" podStartSLOduration=2.013417746 podStartE2EDuration="2.200642967s" totalImagesPullingTime="187.225221ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:15:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:15:07.198357929 +0000 UTC m=+636.182500559" watchObservedRunningTime="2026-07-28 22:15:07.200642967 +0000 UTC m=+636.184785587"
Jul 28 22:15:16.101298 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:16.101252    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:16.101921 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:16.101323    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:16.114712 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:16.114685    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:15:16.220732 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:15:16.220691    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn"
Jul 28 22:16:06.629210 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:06.629175    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7658b4fb4d-ht9hh"]
Jul 28 22:16:06.632753 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:06.632730    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7658b4fb4d-ht9hh"
Jul 28 22:16:06.634950 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:06.634927    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-75hvs\""
Jul 28 22:16:06.635083 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:06.634959    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 28 22:16:06.641253 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:06.641008    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7658b4fb4d-ht9hh"]
Jul 28 22:16:06.762878 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:06.762849    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74jv6\" (UniqueName: \"kubernetes.io/projected/9a821774-325d-46ae-81bc-457b729317fb-kube-api-access-74jv6\") pod \"authorino-7658b4fb4d-ht9hh\" (UID: \"9a821774-325d-46ae-81bc-457b729317fb\") " pod="kuadrant-system/authorino-7658b4fb4d-ht9hh"
Jul 28 22:16:06.763045 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:06.762899    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9a821774-325d-46ae-81bc-457b729317fb-tls-cert\") pod \"authorino-7658b4fb4d-ht9hh\" (UID: \"9a821774-325d-46ae-81bc-457b729317fb\") " pod="kuadrant-system/authorino-7658b4fb4d-ht9hh"
Jul 28 22:16:06.864368 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:06.864328    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74jv6\" (UniqueName: \"kubernetes.io/projected/9a821774-325d-46ae-81bc-457b729317fb-kube-api-access-74jv6\") pod \"authorino-7658b4fb4d-ht9hh\" (UID: \"9a821774-325d-46ae-81bc-457b729317fb\") " pod="kuadrant-system/authorino-7658b4fb4d-ht9hh"
Jul 28 22:16:06.864575 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:06.864383    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9a821774-325d-46ae-81bc-457b729317fb-tls-cert\") pod \"authorino-7658b4fb4d-ht9hh\" (UID: \"9a821774-325d-46ae-81bc-457b729317fb\") " pod="kuadrant-system/authorino-7658b4fb4d-ht9hh"
Jul 28 22:16:06.866851 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:06.866825    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9a821774-325d-46ae-81bc-457b729317fb-tls-cert\") pod \"authorino-7658b4fb4d-ht9hh\" (UID: \"9a821774-325d-46ae-81bc-457b729317fb\") " pod="kuadrant-system/authorino-7658b4fb4d-ht9hh"
Jul 28 22:16:06.872825 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:06.872798    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74jv6\" (UniqueName: \"kubernetes.io/projected/9a821774-325d-46ae-81bc-457b729317fb-kube-api-access-74jv6\") pod \"authorino-7658b4fb4d-ht9hh\" (UID: \"9a821774-325d-46ae-81bc-457b729317fb\") " pod="kuadrant-system/authorino-7658b4fb4d-ht9hh"
Jul 28 22:16:06.944745 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:06.944716    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7658b4fb4d-ht9hh"
Jul 28 22:16:07.087602 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:07.087572    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7658b4fb4d-ht9hh"]
Jul 28 22:16:07.090711 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:16:07.090681    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a821774_325d_46ae_81bc_457b729317fb.slice/crio-9154f84fac51660140f160cb59582dec6aaf1f238fb4cb894596a68416f7e02e WatchSource:0}: Error finding container 9154f84fac51660140f160cb59582dec6aaf1f238fb4cb894596a68416f7e02e: Status 404 returned error can't find the container with id 9154f84fac51660140f160cb59582dec6aaf1f238fb4cb894596a68416f7e02e
Jul 28 22:16:07.451367 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:07.451333    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7658b4fb4d-ht9hh" event={"ID":"9a821774-325d-46ae-81bc-457b729317fb","Type":"ContainerStarted","Data":"9154f84fac51660140f160cb59582dec6aaf1f238fb4cb894596a68416f7e02e"}
Jul 28 22:16:08.457707 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:08.457672    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7658b4fb4d-ht9hh" event={"ID":"9a821774-325d-46ae-81bc-457b729317fb","Type":"ContainerStarted","Data":"d02024f6cc2099b89b2f1e0c3738f8d1704eb2d1b1e76545c8d728a86e9f3d33"}
Jul 28 22:16:08.472544 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:08.472492    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7658b4fb4d-ht9hh" podStartSLOduration=2.013013005 podStartE2EDuration="2.472474023s" totalImagesPullingTime="459.461018ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:16:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:16:08.470455235 +0000 UTC m=+697.454597855" watchObservedRunningTime="2026-07-28 22:16:08.472474023 +0000 UTC m=+697.456616642"
Jul 28 22:16:56.926177 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:56.926138    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-796f547645-mf4pz"]
Jul 28 22:16:56.926742 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:56.926395    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-796f547645-mf4pz" podUID="cf558383-1c95-4f71-9ada-1c9e165f2ecb" containerName="manager" containerID="cri-o://00d375b4e2eae92c96b751abee59b78d7a86893eec9676952b1dedfc421378ba" gracePeriod=30
Jul 28 22:16:57.182768 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:57.182702    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-796f547645-mf4pz"
Jul 28 22:16:57.327225 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:57.327183    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4zsn\" (UniqueName: \"kubernetes.io/projected/cf558383-1c95-4f71-9ada-1c9e165f2ecb-kube-api-access-x4zsn\") pod \"cf558383-1c95-4f71-9ada-1c9e165f2ecb\" (UID: \"cf558383-1c95-4f71-9ada-1c9e165f2ecb\") "
Jul 28 22:16:57.327428 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:57.327259    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf558383-1c95-4f71-9ada-1c9e165f2ecb-webhook-cert\") pod \"cf558383-1c95-4f71-9ada-1c9e165f2ecb\" (UID: \"cf558383-1c95-4f71-9ada-1c9e165f2ecb\") "
Jul 28 22:16:57.329360 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:57.329326    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf558383-1c95-4f71-9ada-1c9e165f2ecb-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "cf558383-1c95-4f71-9ada-1c9e165f2ecb" (UID: "cf558383-1c95-4f71-9ada-1c9e165f2ecb"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:16:57.329485 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:57.329371    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf558383-1c95-4f71-9ada-1c9e165f2ecb-kube-api-access-x4zsn" (OuterVolumeSpecName: "kube-api-access-x4zsn") pod "cf558383-1c95-4f71-9ada-1c9e165f2ecb" (UID: "cf558383-1c95-4f71-9ada-1c9e165f2ecb"). InnerVolumeSpecName "kube-api-access-x4zsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:16:57.428882 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:57.428832    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4zsn\" (UniqueName: \"kubernetes.io/projected/cf558383-1c95-4f71-9ada-1c9e165f2ecb-kube-api-access-x4zsn\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:16:57.428882 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:57.428875    2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf558383-1c95-4f71-9ada-1c9e165f2ecb-webhook-cert\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:16:57.683286 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:57.683227    2574 generic.go:356] "Generic (PLEG): container finished" podID="cf558383-1c95-4f71-9ada-1c9e165f2ecb" containerID="00d375b4e2eae92c96b751abee59b78d7a86893eec9676952b1dedfc421378ba" exitCode=0
Jul 28 22:16:57.683471 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:57.683296    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-796f547645-mf4pz"
Jul 28 22:16:57.683471 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:57.683346    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-796f547645-mf4pz" event={"ID":"cf558383-1c95-4f71-9ada-1c9e165f2ecb","Type":"ContainerDied","Data":"00d375b4e2eae92c96b751abee59b78d7a86893eec9676952b1dedfc421378ba"}
Jul 28 22:16:57.683471 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:57.683397    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-796f547645-mf4pz" event={"ID":"cf558383-1c95-4f71-9ada-1c9e165f2ecb","Type":"ContainerDied","Data":"f33eafdfc76ec0de4c9568187bc1e7041e2b91e2f06bf394c42d3ebd716350ae"}
Jul 28 22:16:57.683471 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:57.683419    2574 scope.go:117] "RemoveContainer" containerID="00d375b4e2eae92c96b751abee59b78d7a86893eec9676952b1dedfc421378ba"
Jul 28 22:16:57.692170 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:57.692151    2574 scope.go:117] "RemoveContainer" containerID="00d375b4e2eae92c96b751abee59b78d7a86893eec9676952b1dedfc421378ba"
Jul 28 22:16:57.692489 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:16:57.692470    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00d375b4e2eae92c96b751abee59b78d7a86893eec9676952b1dedfc421378ba\": container with ID starting with 00d375b4e2eae92c96b751abee59b78d7a86893eec9676952b1dedfc421378ba not found: ID does not exist" containerID="00d375b4e2eae92c96b751abee59b78d7a86893eec9676952b1dedfc421378ba"
Jul 28 22:16:57.692563 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:57.692502    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00d375b4e2eae92c96b751abee59b78d7a86893eec9676952b1dedfc421378ba"} err="failed to get container status \"00d375b4e2eae92c96b751abee59b78d7a86893eec9676952b1dedfc421378ba\": rpc error: code = NotFound desc = could not find container \"00d375b4e2eae92c96b751abee59b78d7a86893eec9676952b1dedfc421378ba\": container with ID starting with 00d375b4e2eae92c96b751abee59b78d7a86893eec9676952b1dedfc421378ba not found: ID does not exist"
Jul 28 22:16:57.699313 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:57.699284    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-796f547645-mf4pz"]
Jul 28 22:16:57.703454 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:57.703427    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-796f547645-mf4pz"]
Jul 28 22:16:59.493940 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:16:59.493906    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf558383-1c95-4f71-9ada-1c9e165f2ecb" path="/var/lib/kubelet/pods/cf558383-1c95-4f71-9ada-1c9e165f2ecb/volumes"
Jul 28 22:17:58.546688 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:17:58.546609    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-796f547645-wbs7b"]
Jul 28 22:17:58.547119 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:17:58.546959    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf558383-1c95-4f71-9ada-1c9e165f2ecb" containerName="manager"
Jul 28 22:17:58.547119 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:17:58.546970    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf558383-1c95-4f71-9ada-1c9e165f2ecb" containerName="manager"
Jul 28 22:17:58.547119 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:17:58.547032    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf558383-1c95-4f71-9ada-1c9e165f2ecb" containerName="manager"
Jul 28 22:17:58.549987 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:17:58.549965    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-796f547645-wbs7b"
Jul 28 22:17:58.552363 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:17:58.552337    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7v8tm\""
Jul 28 22:17:58.552484 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:17:58.552348    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 22:17:58.552484 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:17:58.552369    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 22:17:58.557837 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:17:58.557809    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-796f547645-wbs7b"]
Jul 28 22:17:58.659513 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:17:58.659473    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0cc91484-1bec-4144-ae9c-c491c9a3eb04-webhook-cert\") pod \"maas-controller-796f547645-wbs7b\" (UID: \"0cc91484-1bec-4144-ae9c-c491c9a3eb04\") " pod="opendatahub/maas-controller-796f547645-wbs7b"
Jul 28 22:17:58.659701 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:17:58.659584    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bjhn\" (UniqueName: \"kubernetes.io/projected/0cc91484-1bec-4144-ae9c-c491c9a3eb04-kube-api-access-6bjhn\") pod \"maas-controller-796f547645-wbs7b\" (UID: \"0cc91484-1bec-4144-ae9c-c491c9a3eb04\") " pod="opendatahub/maas-controller-796f547645-wbs7b"
Jul 28 22:17:58.760076 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:17:58.760036    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0cc91484-1bec-4144-ae9c-c491c9a3eb04-webhook-cert\") pod \"maas-controller-796f547645-wbs7b\" (UID: \"0cc91484-1bec-4144-ae9c-c491c9a3eb04\") " pod="opendatahub/maas-controller-796f547645-wbs7b"
Jul 28 22:17:58.760278 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:17:58.760129    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bjhn\" (UniqueName: \"kubernetes.io/projected/0cc91484-1bec-4144-ae9c-c491c9a3eb04-kube-api-access-6bjhn\") pod \"maas-controller-796f547645-wbs7b\" (UID: \"0cc91484-1bec-4144-ae9c-c491c9a3eb04\") " pod="opendatahub/maas-controller-796f547645-wbs7b"
Jul 28 22:17:58.762692 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:17:58.762666    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0cc91484-1bec-4144-ae9c-c491c9a3eb04-webhook-cert\") pod \"maas-controller-796f547645-wbs7b\" (UID: \"0cc91484-1bec-4144-ae9c-c491c9a3eb04\") " pod="opendatahub/maas-controller-796f547645-wbs7b"
Jul 28 22:17:58.767426 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:17:58.767398    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bjhn\" (UniqueName: \"kubernetes.io/projected/0cc91484-1bec-4144-ae9c-c491c9a3eb04-kube-api-access-6bjhn\") pod \"maas-controller-796f547645-wbs7b\" (UID: \"0cc91484-1bec-4144-ae9c-c491c9a3eb04\") " pod="opendatahub/maas-controller-796f547645-wbs7b"
Jul 28 22:17:58.863009 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:17:58.862893    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-796f547645-wbs7b"
Jul 28 22:17:58.998568 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:17:58.998430    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-796f547645-wbs7b"]
Jul 28 22:17:59.001418 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:17:59.001384    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cc91484_1bec_4144_ae9c_c491c9a3eb04.slice/crio-2f322daf27ab6a5c4cd45cc08c1e957d7f5cc3da8c21a69ce50c423025c60afd WatchSource:0}: Error finding container 2f322daf27ab6a5c4cd45cc08c1e957d7f5cc3da8c21a69ce50c423025c60afd: Status 404 returned error can't find the container with id 2f322daf27ab6a5c4cd45cc08c1e957d7f5cc3da8c21a69ce50c423025c60afd
Jul 28 22:17:59.975859 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:17:59.975820    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-796f547645-wbs7b" event={"ID":"0cc91484-1bec-4144-ae9c-c491c9a3eb04","Type":"ContainerStarted","Data":"b7df0677e561a618436ca49b3f8e6c10bf095f7907952d92613bb023d385f800"}
Jul 28 22:17:59.976298 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:17:59.975887    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-796f547645-wbs7b" event={"ID":"0cc91484-1bec-4144-ae9c-c491c9a3eb04","Type":"ContainerStarted","Data":"2f322daf27ab6a5c4cd45cc08c1e957d7f5cc3da8c21a69ce50c423025c60afd"}
Jul 28 22:17:59.976298 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:17:59.976064    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-796f547645-wbs7b"
Jul 28 22:17:59.991524 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:17:59.991475    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-796f547645-wbs7b" podStartSLOduration=1.6198415449999999 podStartE2EDuration="1.991458957s" totalImagesPullingTime="371.617412ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:17:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:17:59.990830238 +0000 UTC m=+808.974972859" watchObservedRunningTime="2026-07-28 22:17:59.991458957 +0000 UTC m=+808.975601576"
Jul 28 22:18:10.982053 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:10.982023    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-796f547645-wbs7b"
Jul 28 22:18:15.671818 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:15.671782    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-796f547645-wbs7b"]
Jul 28 22:18:15.672356 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:15.671998    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-796f547645-wbs7b" podUID="0cc91484-1bec-4144-ae9c-c491c9a3eb04" containerName="manager" containerID="cri-o://b7df0677e561a618436ca49b3f8e6c10bf095f7907952d92613bb023d385f800" gracePeriod=30
Jul 28 22:18:15.920875 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:15.920848    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-796f547645-wbs7b"
Jul 28 22:18:16.006310 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.006274    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bjhn\" (UniqueName: \"kubernetes.io/projected/0cc91484-1bec-4144-ae9c-c491c9a3eb04-kube-api-access-6bjhn\") pod \"0cc91484-1bec-4144-ae9c-c491c9a3eb04\" (UID: \"0cc91484-1bec-4144-ae9c-c491c9a3eb04\") "
Jul 28 22:18:16.006517 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.006345    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0cc91484-1bec-4144-ae9c-c491c9a3eb04-webhook-cert\") pod \"0cc91484-1bec-4144-ae9c-c491c9a3eb04\" (UID: \"0cc91484-1bec-4144-ae9c-c491c9a3eb04\") "
Jul 28 22:18:16.008410 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.008378    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cc91484-1bec-4144-ae9c-c491c9a3eb04-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0cc91484-1bec-4144-ae9c-c491c9a3eb04" (UID: "0cc91484-1bec-4144-ae9c-c491c9a3eb04"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:18:16.008523 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.008478    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cc91484-1bec-4144-ae9c-c491c9a3eb04-kube-api-access-6bjhn" (OuterVolumeSpecName: "kube-api-access-6bjhn") pod "0cc91484-1bec-4144-ae9c-c491c9a3eb04" (UID: "0cc91484-1bec-4144-ae9c-c491c9a3eb04"). InnerVolumeSpecName "kube-api-access-6bjhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:18:16.048519 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.048481    2574 generic.go:356] "Generic (PLEG): container finished" podID="0cc91484-1bec-4144-ae9c-c491c9a3eb04" containerID="b7df0677e561a618436ca49b3f8e6c10bf095f7907952d92613bb023d385f800" exitCode=0
Jul 28 22:18:16.048687 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.048528    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-796f547645-wbs7b"
Jul 28 22:18:16.048687 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.048562    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-796f547645-wbs7b" event={"ID":"0cc91484-1bec-4144-ae9c-c491c9a3eb04","Type":"ContainerDied","Data":"b7df0677e561a618436ca49b3f8e6c10bf095f7907952d92613bb023d385f800"}
Jul 28 22:18:16.048687 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.048621    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-796f547645-wbs7b" event={"ID":"0cc91484-1bec-4144-ae9c-c491c9a3eb04","Type":"ContainerDied","Data":"2f322daf27ab6a5c4cd45cc08c1e957d7f5cc3da8c21a69ce50c423025c60afd"}
Jul 28 22:18:16.048687 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.048649    2574 scope.go:117] "RemoveContainer" containerID="b7df0677e561a618436ca49b3f8e6c10bf095f7907952d92613bb023d385f800"
Jul 28 22:18:16.057909 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.057883    2574 scope.go:117] "RemoveContainer" containerID="b7df0677e561a618436ca49b3f8e6c10bf095f7907952d92613bb023d385f800"
Jul 28 22:18:16.058192 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:18:16.058175    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7df0677e561a618436ca49b3f8e6c10bf095f7907952d92613bb023d385f800\": container with ID starting with b7df0677e561a618436ca49b3f8e6c10bf095f7907952d92613bb023d385f800 not found: ID does not exist" containerID="b7df0677e561a618436ca49b3f8e6c10bf095f7907952d92613bb023d385f800"
Jul 28 22:18:16.058264 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.058202    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7df0677e561a618436ca49b3f8e6c10bf095f7907952d92613bb023d385f800"} err="failed to get container status \"b7df0677e561a618436ca49b3f8e6c10bf095f7907952d92613bb023d385f800\": rpc error: code = NotFound desc = could not find container \"b7df0677e561a618436ca49b3f8e6c10bf095f7907952d92613bb023d385f800\": container with ID starting with b7df0677e561a618436ca49b3f8e6c10bf095f7907952d92613bb023d385f800 not found: ID does not exist"
Jul 28 22:18:16.069215 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.069184    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-796f547645-wbs7b"]
Jul 28 22:18:16.070878 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.070853    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-796f547645-wbs7b"]
Jul 28 22:18:16.107152 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.107120    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bjhn\" (UniqueName: \"kubernetes.io/projected/0cc91484-1bec-4144-ae9c-c491c9a3eb04-kube-api-access-6bjhn\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:18:16.107152 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.107151    2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0cc91484-1bec-4144-ae9c-c491c9a3eb04-webhook-cert\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:18:16.876611 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.876576    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-796f547645-h6r2g"]
Jul 28 22:18:16.876974 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.876920    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cc91484-1bec-4144-ae9c-c491c9a3eb04" containerName="manager"
Jul 28 22:18:16.876974 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.876931    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cc91484-1bec-4144-ae9c-c491c9a3eb04" containerName="manager"
Jul 28 22:18:16.877061 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.876986    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cc91484-1bec-4144-ae9c-c491c9a3eb04" containerName="manager"
Jul 28 22:18:16.881369 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.881347    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-796f547645-h6r2g"
Jul 28 22:18:16.883739 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.883712    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7v8tm\""
Jul 28 22:18:16.883739 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.883720    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 22:18:16.883911 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.883771    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 22:18:16.890462 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.890432    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-796f547645-h6r2g"]
Jul 28 22:18:16.914943 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.914902    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86d33c6f-e796-4e0f-984e-a1703cd034a2-webhook-cert\") pod \"maas-controller-796f547645-h6r2g\" (UID: \"86d33c6f-e796-4e0f-984e-a1703cd034a2\") " pod="opendatahub/maas-controller-796f547645-h6r2g"
Jul 28 22:18:16.914943 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:16.914940    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zpmn\" (UniqueName: \"kubernetes.io/projected/86d33c6f-e796-4e0f-984e-a1703cd034a2-kube-api-access-8zpmn\") pod \"maas-controller-796f547645-h6r2g\" (UID: \"86d33c6f-e796-4e0f-984e-a1703cd034a2\") " pod="opendatahub/maas-controller-796f547645-h6r2g"
Jul 28 22:18:17.015492 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:17.015452    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86d33c6f-e796-4e0f-984e-a1703cd034a2-webhook-cert\") pod \"maas-controller-796f547645-h6r2g\" (UID: \"86d33c6f-e796-4e0f-984e-a1703cd034a2\") " pod="opendatahub/maas-controller-796f547645-h6r2g"
Jul 28 22:18:17.015492 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:17.015494    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zpmn\" (UniqueName: \"kubernetes.io/projected/86d33c6f-e796-4e0f-984e-a1703cd034a2-kube-api-access-8zpmn\") pod \"maas-controller-796f547645-h6r2g\" (UID: \"86d33c6f-e796-4e0f-984e-a1703cd034a2\") " pod="opendatahub/maas-controller-796f547645-h6r2g"
Jul 28 22:18:17.017967 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:17.017946    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86d33c6f-e796-4e0f-984e-a1703cd034a2-webhook-cert\") pod \"maas-controller-796f547645-h6r2g\" (UID: \"86d33c6f-e796-4e0f-984e-a1703cd034a2\") " pod="opendatahub/maas-controller-796f547645-h6r2g"
Jul 28 22:18:17.023679 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:17.023655    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zpmn\" (UniqueName: \"kubernetes.io/projected/86d33c6f-e796-4e0f-984e-a1703cd034a2-kube-api-access-8zpmn\") pod \"maas-controller-796f547645-h6r2g\" (UID: \"86d33c6f-e796-4e0f-984e-a1703cd034a2\") " pod="opendatahub/maas-controller-796f547645-h6r2g"
Jul 28 22:18:17.192573 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:17.192542    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-796f547645-h6r2g"
Jul 28 22:18:17.328743 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:17.328716    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-796f547645-h6r2g"]
Jul 28 22:18:17.331430 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:18:17.331401    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86d33c6f_e796_4e0f_984e_a1703cd034a2.slice/crio-fa370a5941412aba7a66bc9af7d9587820d7092e5e218514eb59c25b3deb16bd WatchSource:0}: Error finding container fa370a5941412aba7a66bc9af7d9587820d7092e5e218514eb59c25b3deb16bd: Status 404 returned error can't find the container with id fa370a5941412aba7a66bc9af7d9587820d7092e5e218514eb59c25b3deb16bd
Jul 28 22:18:17.493917 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:17.493837    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cc91484-1bec-4144-ae9c-c491c9a3eb04" path="/var/lib/kubelet/pods/0cc91484-1bec-4144-ae9c-c491c9a3eb04/volumes"
Jul 28 22:18:18.061070 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:18.060977    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-796f547645-h6r2g" event={"ID":"86d33c6f-e796-4e0f-984e-a1703cd034a2","Type":"ContainerStarted","Data":"cf286f9da3382b913e7dc82261bfdfb3501964203bec05c38b13ac4817afa8ce"}
Jul 28 22:18:18.061070 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:18.061049    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-796f547645-h6r2g"
Jul 28 22:18:18.061529 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:18.061075    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-796f547645-h6r2g" event={"ID":"86d33c6f-e796-4e0f-984e-a1703cd034a2","Type":"ContainerStarted","Data":"fa370a5941412aba7a66bc9af7d9587820d7092e5e218514eb59c25b3deb16bd"}
Jul 28 22:18:18.080614 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:18.080475    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-796f547645-h6r2g" podStartSLOduration=1.675197144 podStartE2EDuration="2.080453903s" totalImagesPullingTime="405.256759ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:18:18.078183108 +0000 UTC m=+827.062325725" watchObservedRunningTime="2026-07-28 22:18:18.080453903 +0000 UTC m=+827.064596523"
Jul 28 22:18:29.068449 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:18:29.068418    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-796f547645-h6r2g"
Jul 28 22:23:32.277400 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:23:32.277360    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6"]
Jul 28 22:23:32.277988 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:23:32.277618    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6" podUID="5cbe3294-4936-496e-bc0d-1cc0c7e1e876" containerName="manager" containerID="cri-o://5638f09ff70100b281bd87ce6fc2ab2a89736b39d837c2711ed609ab5ec78909" gracePeriod=10
Jul 28 22:23:34.330112 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:23:34.330088    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6"
Jul 28 22:23:34.470177 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:23:34.470148    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cbe3294-4936-496e-bc0d-1cc0c7e1e876-extensions-socket-volume\") pod \"5cbe3294-4936-496e-bc0d-1cc0c7e1e876\" (UID: \"5cbe3294-4936-496e-bc0d-1cc0c7e1e876\") "
Jul 28 22:23:34.470359 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:23:34.470192    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xnvw\" (UniqueName: \"kubernetes.io/projected/5cbe3294-4936-496e-bc0d-1cc0c7e1e876-kube-api-access-5xnvw\") pod \"5cbe3294-4936-496e-bc0d-1cc0c7e1e876\" (UID: \"5cbe3294-4936-496e-bc0d-1cc0c7e1e876\") "
Jul 28 22:23:34.470656 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:23:34.470632    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cbe3294-4936-496e-bc0d-1cc0c7e1e876-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5cbe3294-4936-496e-bc0d-1cc0c7e1e876" (UID: "5cbe3294-4936-496e-bc0d-1cc0c7e1e876"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:23:34.472328 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:23:34.472304    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cbe3294-4936-496e-bc0d-1cc0c7e1e876-kube-api-access-5xnvw" (OuterVolumeSpecName: "kube-api-access-5xnvw") pod "5cbe3294-4936-496e-bc0d-1cc0c7e1e876" (UID: "5cbe3294-4936-496e-bc0d-1cc0c7e1e876"). InnerVolumeSpecName "kube-api-access-5xnvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:23:34.545662 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:23:34.545622    2574 generic.go:356] "Generic (PLEG): container finished" podID="5cbe3294-4936-496e-bc0d-1cc0c7e1e876" containerID="5638f09ff70100b281bd87ce6fc2ab2a89736b39d837c2711ed609ab5ec78909" exitCode=0
Jul 28 22:23:34.545662 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:23:34.545662    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6"
Jul 28 22:23:34.545883 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:23:34.545661    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6" event={"ID":"5cbe3294-4936-496e-bc0d-1cc0c7e1e876","Type":"ContainerDied","Data":"5638f09ff70100b281bd87ce6fc2ab2a89736b39d837c2711ed609ab5ec78909"}
Jul 28 22:23:34.545883 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:23:34.545781    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6" event={"ID":"5cbe3294-4936-496e-bc0d-1cc0c7e1e876","Type":"ContainerDied","Data":"0ff63eff59bd89a558f292aece436793a07d23b816b6e5f0c48fbc214a7aba83"}
Jul 28 22:23:34.545883 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:23:34.545803    2574 scope.go:117] "RemoveContainer" containerID="5638f09ff70100b281bd87ce6fc2ab2a89736b39d837c2711ed609ab5ec78909"
Jul 28 22:23:34.557145 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:23:34.557126    2574 scope.go:117] "RemoveContainer" containerID="5638f09ff70100b281bd87ce6fc2ab2a89736b39d837c2711ed609ab5ec78909"
Jul 28 22:23:34.557626 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:23:34.557596    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5638f09ff70100b281bd87ce6fc2ab2a89736b39d837c2711ed609ab5ec78909\": container with ID starting with 5638f09ff70100b281bd87ce6fc2ab2a89736b39d837c2711ed609ab5ec78909 not found: ID does not exist" containerID="5638f09ff70100b281bd87ce6fc2ab2a89736b39d837c2711ed609ab5ec78909"
Jul 28 22:23:34.557722 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:23:34.557639    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5638f09ff70100b281bd87ce6fc2ab2a89736b39d837c2711ed609ab5ec78909"} err="failed to get container status \"5638f09ff70100b281bd87ce6fc2ab2a89736b39d837c2711ed609ab5ec78909\": rpc error: code = NotFound desc = could not find container \"5638f09ff70100b281bd87ce6fc2ab2a89736b39d837c2711ed609ab5ec78909\": container with ID starting with 5638f09ff70100b281bd87ce6fc2ab2a89736b39d837c2711ed609ab5ec78909 not found: ID does not exist"
Jul 28 22:23:34.566650 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:23:34.566627    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6"]
Jul 28 22:23:34.569843 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:23:34.569823    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65vl6"]
Jul 28 22:23:34.571647 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:23:34.571625    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xnvw\" (UniqueName: \"kubernetes.io/projected/5cbe3294-4936-496e-bc0d-1cc0c7e1e876-kube-api-access-5xnvw\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:23:34.571697 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:23:34.571650    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cbe3294-4936-496e-bc0d-1cc0c7e1e876-extensions-socket-volume\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:23:35.493144 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:23:35.493110    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cbe3294-4936-496e-bc0d-1cc0c7e1e876" path="/var/lib/kubelet/pods/5cbe3294-4936-496e-bc0d-1cc0c7e1e876/volumes"
Jul 28 22:24:33.403635 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:24:33.403599    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hxw7b"]
Jul 28 22:24:33.404058 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:24:33.404017    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cbe3294-4936-496e-bc0d-1cc0c7e1e876" containerName="manager"
Jul 28 22:24:33.404058 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:24:33.404029    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cbe3294-4936-496e-bc0d-1cc0c7e1e876" containerName="manager"
Jul 28 22:24:33.404147 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:24:33.404136    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cbe3294-4936-496e-bc0d-1cc0c7e1e876" containerName="manager"
Jul 28 22:24:33.407384 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:24:33.407367    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hxw7b"
Jul 28 22:24:33.409350 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:24:33.409318    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-5fr88\""
Jul 28 22:24:33.419860 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:24:33.419833    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hxw7b"]
Jul 28 22:24:33.484483 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:24:33.484446    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-569fx\" (UniqueName: \"kubernetes.io/projected/51300b59-611c-4d9d-a350-120f6e105afc-kube-api-access-569fx\") pod \"kuadrant-operator-controller-manager-86878c769f-hxw7b\" (UID: \"51300b59-611c-4d9d-a350-120f6e105afc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hxw7b"
Jul 28 22:24:33.484682 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:24:33.484506    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51300b59-611c-4d9d-a350-120f6e105afc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hxw7b\" (UID: \"51300b59-611c-4d9d-a350-120f6e105afc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hxw7b"
Jul 28 22:24:33.585422 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:24:33.585384    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-569fx\" (UniqueName: \"kubernetes.io/projected/51300b59-611c-4d9d-a350-120f6e105afc-kube-api-access-569fx\") pod \"kuadrant-operator-controller-manager-86878c769f-hxw7b\" (UID: \"51300b59-611c-4d9d-a350-120f6e105afc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hxw7b"
Jul 28 22:24:33.585607 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:24:33.585437    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51300b59-611c-4d9d-a350-120f6e105afc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hxw7b\" (UID: \"51300b59-611c-4d9d-a350-120f6e105afc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hxw7b"
Jul 28 22:24:33.585784 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:24:33.585756    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51300b59-611c-4d9d-a350-120f6e105afc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hxw7b\" (UID: \"51300b59-611c-4d9d-a350-120f6e105afc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hxw7b"
Jul 28 22:24:33.595684 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:24:33.595657    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-569fx\" (UniqueName: \"kubernetes.io/projected/51300b59-611c-4d9d-a350-120f6e105afc-kube-api-access-569fx\") pod \"kuadrant-operator-controller-manager-86878c769f-hxw7b\" (UID: \"51300b59-611c-4d9d-a350-120f6e105afc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hxw7b"
Jul 28 22:24:33.717736 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:24:33.717704    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hxw7b"
Jul 28 22:24:33.854566 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:24:33.854534    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hxw7b"]
Jul 28 22:24:33.857490 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:24:33.857461    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51300b59_611c_4d9d_a350_120f6e105afc.slice/crio-16e588b7f1d63c4aefce74d0b8c12f4e4a0afabe389249f7e30f1a6ace45021a WatchSource:0}: Error finding container 16e588b7f1d63c4aefce74d0b8c12f4e4a0afabe389249f7e30f1a6ace45021a: Status 404 returned error can't find the container with id 16e588b7f1d63c4aefce74d0b8c12f4e4a0afabe389249f7e30f1a6ace45021a
Jul 28 22:24:34.829143 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:24:34.829111    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hxw7b" event={"ID":"51300b59-611c-4d9d-a350-120f6e105afc","Type":"ContainerStarted","Data":"a9b2577f933c0a3122a471ddcbc53dffdc3bbee566007e1b0ae50623add68871"}
Jul 28 22:24:34.829569 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:24:34.829161    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hxw7b" event={"ID":"51300b59-611c-4d9d-a350-120f6e105afc","Type":"ContainerStarted","Data":"16e588b7f1d63c4aefce74d0b8c12f4e4a0afabe389249f7e30f1a6ace45021a"}
Jul 28 22:24:34.829569 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:24:34.829201    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hxw7b"
Jul 28 22:24:34.849884 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:24:34.849814    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hxw7b" podStartSLOduration=1.8497970289999999 podStartE2EDuration="1.849797029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:24:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:24:34.848435258 +0000 UTC m=+1203.832577896" watchObservedRunningTime="2026-07-28 22:24:34.849797029 +0000 UTC m=+1203.833939649"
Jul 28 22:24:45.835926 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:24:45.835894    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hxw7b"
Jul 28 22:27:34.605498 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.605458    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"]
Jul 28 22:27:34.608844 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.608825    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:34.610839 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.610822    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-bde4ba45-kserve-self-signed-certs\""
Jul 28 22:27:34.627419 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.627391    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"]
Jul 28 22:27:34.718986 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.718947    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-model-cache\") pod \"e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") " pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:34.719189 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.718995    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzpbh\" (UniqueName: \"kubernetes.io/projected/10a42071-27e7-484d-b175-7ce8662d240c-kube-api-access-hzpbh\") pod \"e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") " pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:34.719189 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.719035    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-dshm\") pod \"e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") " pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:34.719189 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.719102    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10a42071-27e7-484d-b175-7ce8662d240c-tls-certs\") pod \"e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") " pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:34.719189 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.719149    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-home\") pod \"e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") " pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:34.719399 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.719221    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-tmp-dir\") pod \"e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") " pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:34.820582 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.820542    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-model-cache\") pod \"e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") " pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:34.820754 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.820592    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzpbh\" (UniqueName: \"kubernetes.io/projected/10a42071-27e7-484d-b175-7ce8662d240c-kube-api-access-hzpbh\") pod \"e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") " pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:34.820754 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.820621    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-dshm\") pod \"e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") " pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:34.820754 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.820641    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10a42071-27e7-484d-b175-7ce8662d240c-tls-certs\") pod \"e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") " pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:34.820754 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.820665    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-home\") pod \"e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") " pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:34.820754 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.820714    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-tmp-dir\") pod \"e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") " pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:34.821037 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.821005    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-model-cache\") pod \"e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") " pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:34.821106 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.821078    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-tmp-dir\") pod \"e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") " pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:34.821161 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.821136    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-home\") pod \"e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") " pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:34.823014 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.822985    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-dshm\") pod \"e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") " pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:34.823297 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.823277    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10a42071-27e7-484d-b175-7ce8662d240c-tls-certs\") pod \"e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") " pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:34.828279 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.828250    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzpbh\" (UniqueName: \"kubernetes.io/projected/10a42071-27e7-484d-b175-7ce8662d240c-kube-api-access-hzpbh\") pod \"e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") " pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:34.918586 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:34.918496    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:35.058743 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:35.058596    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"]
Jul 28 22:27:35.061874 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:27:35.061822    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10a42071_27e7_484d_b175_7ce8662d240c.slice/crio-d802df0645e7364cfbb696c322134fb6c483b7bbc78fd875206f55ea0fc61cda WatchSource:0}: Error finding container d802df0645e7364cfbb696c322134fb6c483b7bbc78fd875206f55ea0fc61cda: Status 404 returned error can't find the container with id d802df0645e7364cfbb696c322134fb6c483b7bbc78fd875206f55ea0fc61cda
Jul 28 22:27:35.664463 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:35.664423    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l" event={"ID":"10a42071-27e7-484d-b175-7ce8662d240c","Type":"ContainerStarted","Data":"495abe544cf77ca226ba7383c05165aec5ff1032d8daa193dbef111871132a60"}
Jul 28 22:27:35.664463 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:35.664470    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l" event={"ID":"10a42071-27e7-484d-b175-7ce8662d240c","Type":"ContainerStarted","Data":"d802df0645e7364cfbb696c322134fb6c483b7bbc78fd875206f55ea0fc61cda"}
Jul 28 22:27:35.681308 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:35.681252    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l" podStartSLOduration=1.681222173 podStartE2EDuration="1.681222173s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:27:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:27:35.679561893 +0000 UTC m=+1384.663704515" watchObservedRunningTime="2026-07-28 22:27:35.681222173 +0000 UTC m=+1384.665364792"
Jul 28 22:27:44.918871 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:44.918768    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:44.918871 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:44.918820    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:44.931730 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:44.931705    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:45.494178 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.494147    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"]
Jul 28 22:27:45.497809 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.497787    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:45.499538 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.499515    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-bde4ba45-kserve-self-signed-certs\""
Jul 28 22:27:45.506949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.506924    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"]
Jul 28 22:27:45.512245 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.512211    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-tmp-dir\") pod \"e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") " pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:45.512367 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.512279    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-home\") pod \"e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") " pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:45.512367 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.512316    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-model-cache\") pod \"e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") " pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:45.512367 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.512337    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a3d248f-40b7-43d5-83e6-785bf9be5780-tls-certs\") pod \"e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") " pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:45.512479 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.512393    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdxhc\" (UniqueName: \"kubernetes.io/projected/7a3d248f-40b7-43d5-83e6-785bf9be5780-kube-api-access-gdxhc\") pod \"e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") " pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:45.512479 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.512429    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-dshm\") pod \"e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") " pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:45.613011 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.612977    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-home\") pod \"e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") " pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:45.613219 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.613050    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-model-cache\") pod \"e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") " pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:45.613219 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.613084    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a3d248f-40b7-43d5-83e6-785bf9be5780-tls-certs\") pod \"e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") " pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:45.613219 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.613110    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdxhc\" (UniqueName: \"kubernetes.io/projected/7a3d248f-40b7-43d5-83e6-785bf9be5780-kube-api-access-gdxhc\") pod \"e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") " pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:45.613422 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.613287    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-dshm\") pod \"e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") " pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:45.613422 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.613402    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-tmp-dir\") pod \"e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") " pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:45.613537 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.613442    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-home\") pod \"e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") " pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:45.613537 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.613524    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-model-cache\") pod \"e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") " pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:45.613721 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.613698    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-tmp-dir\") pod \"e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") " pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:45.615531 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.615505    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-dshm\") pod \"e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") " pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:45.615625 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.615605    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a3d248f-40b7-43d5-83e6-785bf9be5780-tls-certs\") pod \"e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") " pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:45.621531 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.621508    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdxhc\" (UniqueName: \"kubernetes.io/projected/7a3d248f-40b7-43d5-83e6-785bf9be5780-kube-api-access-gdxhc\") pod \"e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") " pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:45.723262 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.723218    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:27:45.807603 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.807515    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:45.949502 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:45.949432    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"]
Jul 28 22:27:45.952728 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:27:45.952699    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a3d248f_40b7_43d5_83e6_785bf9be5780.slice/crio-99292d20ae735149d4825387d9aa90dceb10637f8643d7ec14b5d65601522c8a WatchSource:0}: Error finding container 99292d20ae735149d4825387d9aa90dceb10637f8643d7ec14b5d65601522c8a: Status 404 returned error can't find the container with id 99292d20ae735149d4825387d9aa90dceb10637f8643d7ec14b5d65601522c8a
Jul 28 22:27:46.717583 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:46.717544    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555" event={"ID":"7a3d248f-40b7-43d5-83e6-785bf9be5780","Type":"ContainerStarted","Data":"9cb8b0114d598acbbaf0d39069fe1eb0c62415ff1038a455b11587914b8e3607"}
Jul 28 22:27:46.717583 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:46.717589    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555" event={"ID":"7a3d248f-40b7-43d5-83e6-785bf9be5780","Type":"ContainerStarted","Data":"99292d20ae735149d4825387d9aa90dceb10637f8643d7ec14b5d65601522c8a"}
Jul 28 22:27:46.731868 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:46.731810    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555" podStartSLOduration=1.731789913 podStartE2EDuration="1.731789913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:27:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:27:46.730788607 +0000 UTC m=+1395.714931227" watchObservedRunningTime="2026-07-28 22:27:46.731789913 +0000 UTC m=+1395.715932536"
Jul 28 22:27:55.808332 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:55.808291    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:55.808829 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:55.808597    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:55.820814 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:55.820782    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:27:56.776458 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:27:56.776425    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:28:06.694818 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:06.694781    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"]
Jul 28 22:28:08.515130 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:08.515094    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"]
Jul 28 22:28:21.714958 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:21.714912    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l" podUID="10a42071-27e7-484d-b175-7ce8662d240c" containerName="main" containerID="cri-o://495abe544cf77ca226ba7383c05165aec5ff1032d8daa193dbef111871132a60" gracePeriod=45
Jul 28 22:28:21.892492 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:21.892298    2574 generic.go:356] "Generic (PLEG): container finished" podID="10a42071-27e7-484d-b175-7ce8662d240c" containerID="495abe544cf77ca226ba7383c05165aec5ff1032d8daa193dbef111871132a60" exitCode=0
Jul 28 22:28:21.892492 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:21.892432    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l" event={"ID":"10a42071-27e7-484d-b175-7ce8662d240c","Type":"ContainerDied","Data":"495abe544cf77ca226ba7383c05165aec5ff1032d8daa193dbef111871132a60"}
Jul 28 22:28:22.010896 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.010864    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:28:22.152221 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.152181    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10a42071-27e7-484d-b175-7ce8662d240c-tls-certs\") pod \"10a42071-27e7-484d-b175-7ce8662d240c\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") "
Jul 28 22:28:22.152402 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.152261    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-tmp-dir\") pod \"10a42071-27e7-484d-b175-7ce8662d240c\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") "
Jul 28 22:28:22.152402 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.152315    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzpbh\" (UniqueName: \"kubernetes.io/projected/10a42071-27e7-484d-b175-7ce8662d240c-kube-api-access-hzpbh\") pod \"10a42071-27e7-484d-b175-7ce8662d240c\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") "
Jul 28 22:28:22.152402 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.152366    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-model-cache\") pod \"10a42071-27e7-484d-b175-7ce8662d240c\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") "
Jul 28 22:28:22.152402 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.152399    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-dshm\") pod \"10a42071-27e7-484d-b175-7ce8662d240c\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") "
Jul 28 22:28:22.152609 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.152471    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-home\") pod \"10a42071-27e7-484d-b175-7ce8662d240c\" (UID: \"10a42071-27e7-484d-b175-7ce8662d240c\") "
Jul 28 22:28:22.152609 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.152544    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "10a42071-27e7-484d-b175-7ce8662d240c" (UID: "10a42071-27e7-484d-b175-7ce8662d240c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:28:22.152794 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.152722    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-model-cache" (OuterVolumeSpecName: "model-cache") pod "10a42071-27e7-484d-b175-7ce8662d240c" (UID: "10a42071-27e7-484d-b175-7ce8662d240c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:28:22.152794 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.152760    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-home" (OuterVolumeSpecName: "home") pod "10a42071-27e7-484d-b175-7ce8662d240c" (UID: "10a42071-27e7-484d-b175-7ce8662d240c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:28:22.152996 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.152894    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-home\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:28:22.152996 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.152912    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-tmp-dir\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:28:22.152996 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.152925    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-model-cache\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:28:22.154981 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.154943    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10a42071-27e7-484d-b175-7ce8662d240c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "10a42071-27e7-484d-b175-7ce8662d240c" (UID: "10a42071-27e7-484d-b175-7ce8662d240c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:28:22.155101 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.154991    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10a42071-27e7-484d-b175-7ce8662d240c-kube-api-access-hzpbh" (OuterVolumeSpecName: "kube-api-access-hzpbh") pod "10a42071-27e7-484d-b175-7ce8662d240c" (UID: "10a42071-27e7-484d-b175-7ce8662d240c"). InnerVolumeSpecName "kube-api-access-hzpbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:28:22.156835 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.156812    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-dshm" (OuterVolumeSpecName: "dshm") pod "10a42071-27e7-484d-b175-7ce8662d240c" (UID: "10a42071-27e7-484d-b175-7ce8662d240c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:28:22.254492 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.254406    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzpbh\" (UniqueName: \"kubernetes.io/projected/10a42071-27e7-484d-b175-7ce8662d240c-kube-api-access-hzpbh\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:28:22.254492 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.254439    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10a42071-27e7-484d-b175-7ce8662d240c-dshm\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:28:22.254492 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.254449    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10a42071-27e7-484d-b175-7ce8662d240c-tls-certs\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:28:22.898618 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.898588    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"
Jul 28 22:28:22.898618 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.898596    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l" event={"ID":"10a42071-27e7-484d-b175-7ce8662d240c","Type":"ContainerDied","Data":"d802df0645e7364cfbb696c322134fb6c483b7bbc78fd875206f55ea0fc61cda"}
Jul 28 22:28:22.899146 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.898643    2574 scope.go:117] "RemoveContainer" containerID="495abe544cf77ca226ba7383c05165aec5ff1032d8daa193dbef111871132a60"
Jul 28 22:28:22.919451 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.919425    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"]
Jul 28 22:28:22.923349 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:22.923326    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-bde4ba45-kserve-7bb7dcf5f9-6z55l"]
Jul 28 22:28:23.494189 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.494156    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10a42071-27e7-484d-b175-7ce8662d240c" path="/var/lib/kubelet/pods/10a42071-27e7-484d-b175-7ce8662d240c/volumes"
Jul 28 22:28:23.534374 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.534326    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555" podUID="7a3d248f-40b7-43d5-83e6-785bf9be5780" containerName="main" containerID="cri-o://9cb8b0114d598acbbaf0d39069fe1eb0c62415ff1038a455b11587914b8e3607" gracePeriod=45
Jul 28 22:28:23.805342 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.805313    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:28:23.905116 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.905081    2574 generic.go:356] "Generic (PLEG): container finished" podID="7a3d248f-40b7-43d5-83e6-785bf9be5780" containerID="9cb8b0114d598acbbaf0d39069fe1eb0c62415ff1038a455b11587914b8e3607" exitCode=0
Jul 28 22:28:23.905548 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.905143    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"
Jul 28 22:28:23.905548 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.905168    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555" event={"ID":"7a3d248f-40b7-43d5-83e6-785bf9be5780","Type":"ContainerDied","Data":"9cb8b0114d598acbbaf0d39069fe1eb0c62415ff1038a455b11587914b8e3607"}
Jul 28 22:28:23.905548 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.905212    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555" event={"ID":"7a3d248f-40b7-43d5-83e6-785bf9be5780","Type":"ContainerDied","Data":"99292d20ae735149d4825387d9aa90dceb10637f8643d7ec14b5d65601522c8a"}
Jul 28 22:28:23.905548 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.905258    2574 scope.go:117] "RemoveContainer" containerID="9cb8b0114d598acbbaf0d39069fe1eb0c62415ff1038a455b11587914b8e3607"
Jul 28 22:28:23.913829 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.913803    2574 scope.go:117] "RemoveContainer" containerID="9cb8b0114d598acbbaf0d39069fe1eb0c62415ff1038a455b11587914b8e3607"
Jul 28 22:28:23.914113 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:28:23.914087    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cb8b0114d598acbbaf0d39069fe1eb0c62415ff1038a455b11587914b8e3607\": container with ID starting with 9cb8b0114d598acbbaf0d39069fe1eb0c62415ff1038a455b11587914b8e3607 not found: ID does not exist" containerID="9cb8b0114d598acbbaf0d39069fe1eb0c62415ff1038a455b11587914b8e3607"
Jul 28 22:28:23.914168 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.914124    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cb8b0114d598acbbaf0d39069fe1eb0c62415ff1038a455b11587914b8e3607"} err="failed to get container status \"9cb8b0114d598acbbaf0d39069fe1eb0c62415ff1038a455b11587914b8e3607\": rpc error: code = NotFound desc = could not find container \"9cb8b0114d598acbbaf0d39069fe1eb0c62415ff1038a455b11587914b8e3607\": container with ID starting with 9cb8b0114d598acbbaf0d39069fe1eb0c62415ff1038a455b11587914b8e3607 not found: ID does not exist"
Jul 28 22:28:23.967856 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.967820    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-home\") pod \"7a3d248f-40b7-43d5-83e6-785bf9be5780\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") "
Jul 28 22:28:23.967856 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.967857    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-dshm\") pod \"7a3d248f-40b7-43d5-83e6-785bf9be5780\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") "
Jul 28 22:28:23.968103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.967921    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdxhc\" (UniqueName: \"kubernetes.io/projected/7a3d248f-40b7-43d5-83e6-785bf9be5780-kube-api-access-gdxhc\") pod \"7a3d248f-40b7-43d5-83e6-785bf9be5780\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") "
Jul 28 22:28:23.968103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.967939    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-model-cache\") pod \"7a3d248f-40b7-43d5-83e6-785bf9be5780\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") "
Jul 28 22:28:23.968103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.967986    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a3d248f-40b7-43d5-83e6-785bf9be5780-tls-certs\") pod \"7a3d248f-40b7-43d5-83e6-785bf9be5780\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") "
Jul 28 22:28:23.968103 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.968027    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-tmp-dir\") pod \"7a3d248f-40b7-43d5-83e6-785bf9be5780\" (UID: \"7a3d248f-40b7-43d5-83e6-785bf9be5780\") "
Jul 28 22:28:23.968334 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.968142    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-home" (OuterVolumeSpecName: "home") pod "7a3d248f-40b7-43d5-83e6-785bf9be5780" (UID: "7a3d248f-40b7-43d5-83e6-785bf9be5780"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:28:23.968334 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.968227    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-model-cache" (OuterVolumeSpecName: "model-cache") pod "7a3d248f-40b7-43d5-83e6-785bf9be5780" (UID: "7a3d248f-40b7-43d5-83e6-785bf9be5780"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:28:23.968455 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.968333    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-home\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:28:23.968455 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.968399    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7a3d248f-40b7-43d5-83e6-785bf9be5780" (UID: "7a3d248f-40b7-43d5-83e6-785bf9be5780"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:28:23.970164 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.970141    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-dshm" (OuterVolumeSpecName: "dshm") pod "7a3d248f-40b7-43d5-83e6-785bf9be5780" (UID: "7a3d248f-40b7-43d5-83e6-785bf9be5780"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:28:23.970273 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.970164    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a3d248f-40b7-43d5-83e6-785bf9be5780-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7a3d248f-40b7-43d5-83e6-785bf9be5780" (UID: "7a3d248f-40b7-43d5-83e6-785bf9be5780"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:28:23.970315 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:23.970283    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a3d248f-40b7-43d5-83e6-785bf9be5780-kube-api-access-gdxhc" (OuterVolumeSpecName: "kube-api-access-gdxhc") pod "7a3d248f-40b7-43d5-83e6-785bf9be5780" (UID: "7a3d248f-40b7-43d5-83e6-785bf9be5780"). InnerVolumeSpecName "kube-api-access-gdxhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:28:24.069562 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:24.069467    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-tmp-dir\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:28:24.069562 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:24.069506    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-dshm\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:28:24.069562 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:24.069517    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdxhc\" (UniqueName: \"kubernetes.io/projected/7a3d248f-40b7-43d5-83e6-785bf9be5780-kube-api-access-gdxhc\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:28:24.069562 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:24.069526    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a3d248f-40b7-43d5-83e6-785bf9be5780-model-cache\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:28:24.069562 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:24.069534    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a3d248f-40b7-43d5-83e6-785bf9be5780-tls-certs\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:28:24.225596 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:24.225563    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"]
Jul 28 22:28:24.230955 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:24.230923    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-bde4ba45-kserve-7b669589f9-vx555"]
Jul 28 22:28:25.494050 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:28:25.494014    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a3d248f-40b7-43d5-83e6-785bf9be5780" path="/var/lib/kubelet/pods/7a3d248f-40b7-43d5-83e6-785bf9be5780/volumes"
Jul 28 22:32:01.181742 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.181711    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"]
Jul 28 22:32:01.182213 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.182091    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a3d248f-40b7-43d5-83e6-785bf9be5780" containerName="main"
Jul 28 22:32:01.182213 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.182102    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a3d248f-40b7-43d5-83e6-785bf9be5780" containerName="main"
Jul 28 22:32:01.182213 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.182111    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10a42071-27e7-484d-b175-7ce8662d240c" containerName="main"
Jul 28 22:32:01.182213 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.182116    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="10a42071-27e7-484d-b175-7ce8662d240c" containerName="main"
Jul 28 22:32:01.182213 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.182170    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a3d248f-40b7-43d5-83e6-785bf9be5780" containerName="main"
Jul 28 22:32:01.182213 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.182178    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="10a42071-27e7-484d-b175-7ce8662d240c" containerName="main"
Jul 28 22:32:01.185213 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.185196    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.187258 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.187207    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-09e3e515-openshift-default-dockercfg-7h587\""
Jul 28 22:32:01.203695 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.203662    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"]
Jul 28 22:32:01.254287 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.254246    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istiod-ca-cert\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.254457 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.254293    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-podinfo\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.254457 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.254317    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-workload-socket\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.254457 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.254336    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-workload-certs\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.254457 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.254380    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-token\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.254457 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.254435    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-envoy\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.254619 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.254463    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-credential-socket\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.254619 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.254497    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-data\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.254619 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.254572    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rv7q\" (UniqueName: \"kubernetes.io/projected/eff0e18d-b03a-446a-b7c0-b294d6c84d50-kube-api-access-5rv7q\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.355947 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.355912    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-envoy\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.355947 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.355953    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-credential-socket\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.356173 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.355971    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-data\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.356173 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.356032    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rv7q\" (UniqueName: \"kubernetes.io/projected/eff0e18d-b03a-446a-b7c0-b294d6c84d50-kube-api-access-5rv7q\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.356173 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.356072    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istiod-ca-cert\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.356173 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.356109    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-podinfo\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.356173 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.356142    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-workload-socket\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.356173 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.356166    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-workload-certs\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.356494 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.356454    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-credential-socket\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.356555 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.356529    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-data\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.356609 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.356583    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-workload-socket\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.356662 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.356611    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-workload-certs\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.356715 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.356656    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-token\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.357149 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.357121    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istiod-ca-cert\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.358895 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.358875    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-envoy\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.359001 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.358951    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-podinfo\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.365414 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.365388    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-token\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.365595 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.365575    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rv7q\" (UniqueName: \"kubernetes.io/projected/eff0e18d-b03a-446a-b7c0-b294d6c84d50-kube-api-access-5rv7q\") pod \"e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") " pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.494665 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.494631    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:01.638916 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.638877    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"]
Jul 28 22:32:01.641801 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:32:01.641772    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeff0e18d_b03a_446a_b7c0_b294d6c84d50.slice/crio-e09052ec1b90ee202028e02db773026be0907082439e69277d18153dd5cac2a3 WatchSource:0}: Error finding container e09052ec1b90ee202028e02db773026be0907082439e69277d18153dd5cac2a3: Status 404 returned error can't find the container with id e09052ec1b90ee202028e02db773026be0907082439e69277d18153dd5cac2a3
Jul 28 22:32:01.644122 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.644092    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:32:01.644208 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.644159    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:32:01.644208 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.644184    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:32:01.942756 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.942715    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv" event={"ID":"eff0e18d-b03a-446a-b7c0-b294d6c84d50","Type":"ContainerStarted","Data":"ec2d978f648adea8e67a5bc739419f6c6bb7d8172f7919f544b77faba5350b58"}
Jul 28 22:32:01.942934 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.942789    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv" event={"ID":"eff0e18d-b03a-446a-b7c0-b294d6c84d50","Type":"ContainerStarted","Data":"e09052ec1b90ee202028e02db773026be0907082439e69277d18153dd5cac2a3"}
Jul 28 22:32:01.975790 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:01.975736    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv" podStartSLOduration=0.975719741 podStartE2EDuration="975.719741ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:32:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:32:01.974006789 +0000 UTC m=+1650.958149409" watchObservedRunningTime="2026-07-28 22:32:01.975719741 +0000 UTC m=+1650.959862433"
Jul 28 22:32:02.495105 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:02.495062    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:02.500097 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:02.500071    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:02.948293 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:02.948263    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:02.949396 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:02.949373    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:03.497488 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:03.497457    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj"]
Jul 28 22:32:03.500524 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:03.500503    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj"
Jul 28 22:32:03.502652 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:03.502634    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-09e3e515-serving-cert\""
Jul 28 22:32:03.512856 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:03.512832    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj"]
Jul 28 22:32:03.579517 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:03.579478    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkwr2\" (UniqueName: \"kubernetes.io/projected/d62edc50-135c-47d6-a905-4c3530e66a40-kube-api-access-rkwr2\") pod \"maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj\" (UID: \"d62edc50-135c-47d6-a905-4c3530e66a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj"
Jul 28 22:32:03.579517 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:03.579522    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d62edc50-135c-47d6-a905-4c3530e66a40-openshift-service-ca\") pod \"maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj\" (UID: \"d62edc50-135c-47d6-a905-4c3530e66a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj"
Jul 28 22:32:03.579768 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:03.579583    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d62edc50-135c-47d6-a905-4c3530e66a40-maas-api-tls\") pod \"maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj\" (UID: \"d62edc50-135c-47d6-a905-4c3530e66a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj"
Jul 28 22:32:03.681029 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:03.680991    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkwr2\" (UniqueName: \"kubernetes.io/projected/d62edc50-135c-47d6-a905-4c3530e66a40-kube-api-access-rkwr2\") pod \"maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj\" (UID: \"d62edc50-135c-47d6-a905-4c3530e66a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj"
Jul 28 22:32:03.681029 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:03.681029    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d62edc50-135c-47d6-a905-4c3530e66a40-openshift-service-ca\") pod \"maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj\" (UID: \"d62edc50-135c-47d6-a905-4c3530e66a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj"
Jul 28 22:32:03.681355 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:03.681054    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d62edc50-135c-47d6-a905-4c3530e66a40-maas-api-tls\") pod \"maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj\" (UID: \"d62edc50-135c-47d6-a905-4c3530e66a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj"
Jul 28 22:32:03.681697 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:03.681678    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d62edc50-135c-47d6-a905-4c3530e66a40-openshift-service-ca\") pod \"maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj\" (UID: \"d62edc50-135c-47d6-a905-4c3530e66a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj"
Jul 28 22:32:03.683587 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:03.683571    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d62edc50-135c-47d6-a905-4c3530e66a40-maas-api-tls\") pod \"maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj\" (UID: \"d62edc50-135c-47d6-a905-4c3530e66a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj"
Jul 28 22:32:03.690663 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:03.690636    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkwr2\" (UniqueName: \"kubernetes.io/projected/d62edc50-135c-47d6-a905-4c3530e66a40-kube-api-access-rkwr2\") pod \"maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj\" (UID: \"d62edc50-135c-47d6-a905-4c3530e66a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj"
Jul 28 22:32:03.815625 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:03.815535    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj"
Jul 28 22:32:03.955110 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:03.955084    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj"]
Jul 28 22:32:03.957412 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:32:03.957387    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd62edc50_135c_47d6_a905_4c3530e66a40.slice/crio-b559ebbc6fe6690ac00b751356fb80c29ab660360fcd7e433219de9f6792e470 WatchSource:0}: Error finding container b559ebbc6fe6690ac00b751356fb80c29ab660360fcd7e433219de9f6792e470: Status 404 returned error can't find the container with id b559ebbc6fe6690ac00b751356fb80c29ab660360fcd7e433219de9f6792e470
Jul 28 22:32:03.959196 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:03.959178    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 22:32:04.959517 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:04.959475    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj" event={"ID":"d62edc50-135c-47d6-a905-4c3530e66a40","Type":"ContainerStarted","Data":"8364733bd920771aac6b67825278d78e6c9e681ad4349de0a6e780c19d5037d7"}
Jul 28 22:32:04.959938 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:04.959578    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj" event={"ID":"d62edc50-135c-47d6-a905-4c3530e66a40","Type":"ContainerStarted","Data":"b559ebbc6fe6690ac00b751356fb80c29ab660360fcd7e433219de9f6792e470"}
Jul 28 22:32:04.977677 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:04.977588    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj" podStartSLOduration=1.223346688 podStartE2EDuration="1.977572353s" totalImagesPullingTime="754.225665ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:32:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:32:04.975411533 +0000 UTC m=+1653.959554155" watchObservedRunningTime="2026-07-28 22:32:04.977572353 +0000 UTC m=+1653.961714973"
Jul 28 22:32:05.965042 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:05.965009    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj"
Jul 28 22:32:11.975927 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:11.975893    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj"
Jul 28 22:32:17.020907 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:17.020871    2574 generic.go:356] "Generic (PLEG): container finished" podID="eff0e18d-b03a-446a-b7c0-b294d6c84d50" containerID="ec2d978f648adea8e67a5bc739419f6c6bb7d8172f7919f544b77faba5350b58" exitCode=0
Jul 28 22:32:17.021363 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:17.020943    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv" event={"ID":"eff0e18d-b03a-446a-b7c0-b294d6c84d50","Type":"ContainerDied","Data":"ec2d978f648adea8e67a5bc739419f6c6bb7d8172f7919f544b77faba5350b58"}
Jul 28 22:32:17.021417 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:17.021385    2574 scope.go:117] "RemoveContainer" containerID="ec2d978f648adea8e67a5bc739419f6c6bb7d8172f7919f544b77faba5350b58"
Jul 28 22:32:17.022429 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:17.022400    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:32:17.022502 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:17.022483    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:32:17.022548 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:17.022527    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:32:17.948439 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:17.948369    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:18.027892 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:18.027857    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv" event={"ID":"eff0e18d-b03a-446a-b7c0-b294d6c84d50","Type":"ContainerStarted","Data":"265b5549b1d64259a503699ce8bd167bb3176bf9b7ba994250f6f3caa74826c2"}
Jul 28 22:32:18.495857 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:18.495819    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:18.497062 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:18.497026    2574 patch_prober.go:28] interesting pod/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 28 22:32:18.497219 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:18.497076    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv" podUID="eff0e18d-b03a-446a-b7c0-b294d6c84d50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 28 22:32:19.495604 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:19.495569    2574 patch_prober.go:28] interesting pod/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 28 22:32:19.496008 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:19.495619    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv" podUID="eff0e18d-b03a-446a-b7c0-b294d6c84d50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 28 22:32:20.495858 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:20.495825    2574 patch_prober.go:28] interesting pod/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 28 22:32:20.496296 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:20.495878    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv" podUID="eff0e18d-b03a-446a-b7c0-b294d6c84d50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 28 22:32:21.495895 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:21.495858    2574 patch_prober.go:28] interesting pod/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 28 22:32:21.496351 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:21.495906    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv" podUID="eff0e18d-b03a-446a-b7c0-b294d6c84d50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 28 22:32:22.499050 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:22.499022    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:22.499583 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:22.499528    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:22.500426 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:22.500402    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:33.962890 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:33.962211    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj"]
Jul 28 22:32:33.963845 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:33.963805    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj" podUID="d62edc50-135c-47d6-a905-4c3530e66a40" containerName="maas-api" containerID="cri-o://8364733bd920771aac6b67825278d78e6c9e681ad4349de0a6e780c19d5037d7" gracePeriod=30
Jul 28 22:32:34.110059 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:34.110026    2574 generic.go:356] "Generic (PLEG): container finished" podID="d62edc50-135c-47d6-a905-4c3530e66a40" containerID="8364733bd920771aac6b67825278d78e6c9e681ad4349de0a6e780c19d5037d7" exitCode=0
Jul 28 22:32:34.110277 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:34.110098    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj" event={"ID":"d62edc50-135c-47d6-a905-4c3530e66a40","Type":"ContainerDied","Data":"8364733bd920771aac6b67825278d78e6c9e681ad4349de0a6e780c19d5037d7"}
Jul 28 22:32:34.229514 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:34.229487    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj"
Jul 28 22:32:34.285994 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:34.285956    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"]
Jul 28 22:32:34.286243 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:34.286206    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv" podUID="eff0e18d-b03a-446a-b7c0-b294d6c84d50" containerName="istio-proxy" containerID="cri-o://265b5549b1d64259a503699ce8bd167bb3176bf9b7ba994250f6f3caa74826c2" gracePeriod=30
Jul 28 22:32:34.363807 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:34.363771    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d62edc50-135c-47d6-a905-4c3530e66a40-openshift-service-ca\") pod \"d62edc50-135c-47d6-a905-4c3530e66a40\" (UID: \"d62edc50-135c-47d6-a905-4c3530e66a40\") "
Jul 28 22:32:34.363990 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:34.363828    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkwr2\" (UniqueName: \"kubernetes.io/projected/d62edc50-135c-47d6-a905-4c3530e66a40-kube-api-access-rkwr2\") pod \"d62edc50-135c-47d6-a905-4c3530e66a40\" (UID: \"d62edc50-135c-47d6-a905-4c3530e66a40\") "
Jul 28 22:32:34.363990 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:34.363866    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d62edc50-135c-47d6-a905-4c3530e66a40-maas-api-tls\") pod \"d62edc50-135c-47d6-a905-4c3530e66a40\" (UID: \"d62edc50-135c-47d6-a905-4c3530e66a40\") "
Jul 28 22:32:34.364188 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:34.364162    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d62edc50-135c-47d6-a905-4c3530e66a40-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d62edc50-135c-47d6-a905-4c3530e66a40" (UID: "d62edc50-135c-47d6-a905-4c3530e66a40"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:32:34.366118 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:34.366095    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d62edc50-135c-47d6-a905-4c3530e66a40-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d62edc50-135c-47d6-a905-4c3530e66a40" (UID: "d62edc50-135c-47d6-a905-4c3530e66a40"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:32:34.366248 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:34.366214    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d62edc50-135c-47d6-a905-4c3530e66a40-kube-api-access-rkwr2" (OuterVolumeSpecName: "kube-api-access-rkwr2") pod "d62edc50-135c-47d6-a905-4c3530e66a40" (UID: "d62edc50-135c-47d6-a905-4c3530e66a40"). InnerVolumeSpecName "kube-api-access-rkwr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:32:34.465442 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:34.465360    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d62edc50-135c-47d6-a905-4c3530e66a40-openshift-service-ca\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:32:34.465442 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:34.465386    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkwr2\" (UniqueName: \"kubernetes.io/projected/d62edc50-135c-47d6-a905-4c3530e66a40-kube-api-access-rkwr2\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:32:34.465442 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:34.465396    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d62edc50-135c-47d6-a905-4c3530e66a40-maas-api-tls\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:32:35.116093 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.116023    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj"
Jul 28 22:32:35.116517 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.116065    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj" event={"ID":"d62edc50-135c-47d6-a905-4c3530e66a40","Type":"ContainerDied","Data":"b559ebbc6fe6690ac00b751356fb80c29ab660360fcd7e433219de9f6792e470"}
Jul 28 22:32:35.116517 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.116191    2574 scope.go:117] "RemoveContainer" containerID="8364733bd920771aac6b67825278d78e6c9e681ad4349de0a6e780c19d5037d7"
Jul 28 22:32:35.137193 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.137158    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj"]
Jul 28 22:32:35.140247 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.140205    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-09e3e515-74d8d46857-n2tdj"]
Jul 28 22:32:35.262361 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.262325    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"]
Jul 28 22:32:35.262705 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.262692    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d62edc50-135c-47d6-a905-4c3530e66a40" containerName="maas-api"
Jul 28 22:32:35.262705 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.262706    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d62edc50-135c-47d6-a905-4c3530e66a40" containerName="maas-api"
Jul 28 22:32:35.262800 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.262781    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d62edc50-135c-47d6-a905-4c3530e66a40" containerName="maas-api"
Jul 28 22:32:35.265661 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.265641    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.267607 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.267585    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-6a50116a-gw-openshift-default-dockercfg-kq55m\""
Jul 28 22:32:35.276022 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.275998    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"]
Jul 28 22:32:35.374535 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.374429    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-envoy\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.374535 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.374471    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-podinfo\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.374535 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.374504    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt6k7\" (UniqueName: \"kubernetes.io/projected/58ac102e-4961-4250-a49f-bafec0dc3e3d-kube-api-access-dt6k7\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.374839 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.374558    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58ac102e-4961-4250-a49f-bafec0dc3e3d-istiod-ca-cert\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.374839 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.374621    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-token\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.374839 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.374657    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-workload-socket\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.374839 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.374712    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-credential-socket\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.374839 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.374751    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-data\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.374839 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.374778    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-workload-certs\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.475469 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.475431    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58ac102e-4961-4250-a49f-bafec0dc3e3d-istiod-ca-cert\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.475679 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.475492    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-token\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.475679 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.475615    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-workload-socket\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.475813 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.475702    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-credential-socket\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.475813 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.475746    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-data\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.475813 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.475791    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-workload-certs\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.475975 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.475856    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-envoy\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.475975 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.475887    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-podinfo\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.475975 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.475921    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt6k7\" (UniqueName: \"kubernetes.io/projected/58ac102e-4961-4250-a49f-bafec0dc3e3d-kube-api-access-dt6k7\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.476178 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.475972    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-workload-socket\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.476178 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.476054    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-credential-socket\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.476178 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.476146    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-workload-certs\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.476178 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.476153    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58ac102e-4961-4250-a49f-bafec0dc3e3d-istiod-ca-cert\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.476453 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.476266    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-data\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.478134 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.478112    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-envoy\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.478450 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.478430    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-podinfo\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.493092 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.493065    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-token\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.493595 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.493574    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt6k7\" (UniqueName: \"kubernetes.io/projected/58ac102e-4961-4250-a49f-bafec0dc3e3d-kube-api-access-dt6k7\") pod \"e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") " pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.495327 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.495306    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d62edc50-135c-47d6-a905-4c3530e66a40" path="/var/lib/kubelet/pods/d62edc50-135c-47d6-a905-4c3530e66a40/volumes"
Jul 28 22:32:35.577005 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.576971    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:35.752449 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.752421    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"]
Jul 28 22:32:35.754538 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:32:35.754511    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58ac102e_4961_4250_a49f_bafec0dc3e3d.slice/crio-8c1d43be1b859b11f1b1958e1f7fcde48667fbdf0206a64313015332fdcc8377 WatchSource:0}: Error finding container 8c1d43be1b859b11f1b1958e1f7fcde48667fbdf0206a64313015332fdcc8377: Status 404 returned error can't find the container with id 8c1d43be1b859b11f1b1958e1f7fcde48667fbdf0206a64313015332fdcc8377
Jul 28 22:32:35.756719 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.756688    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:32:35.756790 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.756755    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:32:35.756790 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:35.756783    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:32:36.122855 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:36.122820    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh" event={"ID":"58ac102e-4961-4250-a49f-bafec0dc3e3d","Type":"ContainerStarted","Data":"952efef98968b934cbff69e9edb4385caa27f844a650cfc612b09b5d932efc9c"}
Jul 28 22:32:36.123365 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:36.122877    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh" event={"ID":"58ac102e-4961-4250-a49f-bafec0dc3e3d","Type":"ContainerStarted","Data":"8c1d43be1b859b11f1b1958e1f7fcde48667fbdf0206a64313015332fdcc8377"}
Jul 28 22:32:36.143497 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:36.143430    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh" podStartSLOduration=1.1434082 podStartE2EDuration="1.1434082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:32:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:32:36.140993795 +0000 UTC m=+1685.125136416" watchObservedRunningTime="2026-07-28 22:32:36.1434082 +0000 UTC m=+1685.127550822"
Jul 28 22:32:36.578162 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:36.578119    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:36.583067 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:36.583035    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:37.128368 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:37.128339    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:37.129214 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:37.129195    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:37.502157 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:37.502119    2574 patch_prober.go:28] interesting pod/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 22:32:37.502351 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:37.502184    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv" podUID="eff0e18d-b03a-446a-b7c0-b294d6c84d50" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 22:32:39.559658 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.559583    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:39.614633 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.614606    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-workload-certs\") pod \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") "
Jul 28 22:32:39.614832 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.614671    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-token\") pod \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") "
Jul 28 22:32:39.614832 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.614701    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-workload-socket\") pod \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") "
Jul 28 22:32:39.614832 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.614743    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-data\") pod \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") "
Jul 28 22:32:39.614832 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.614785    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istiod-ca-cert\") pod \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") "
Jul 28 22:32:39.614832 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.614810    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-podinfo\") pod \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") "
Jul 28 22:32:39.615111 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.614855    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-envoy\") pod \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") "
Jul 28 22:32:39.615111 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.614862    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eff0e18d-b03a-446a-b7c0-b294d6c84d50" (UID: "eff0e18d-b03a-446a-b7c0-b294d6c84d50"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:32:39.615111 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.614881    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rv7q\" (UniqueName: \"kubernetes.io/projected/eff0e18d-b03a-446a-b7c0-b294d6c84d50-kube-api-access-5rv7q\") pod \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") "
Jul 28 22:32:39.615111 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.614932    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-credential-socket\") pod \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\" (UID: \"eff0e18d-b03a-446a-b7c0-b294d6c84d50\") "
Jul 28 22:32:39.615111 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.615046    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eff0e18d-b03a-446a-b7c0-b294d6c84d50" (UID: "eff0e18d-b03a-446a-b7c0-b294d6c84d50"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:32:39.615420 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.615191    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eff0e18d-b03a-446a-b7c0-b294d6c84d50" (UID: "eff0e18d-b03a-446a-b7c0-b294d6c84d50"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:32:39.616081 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.615511    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-workload-certs\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:32:39.616081 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.615537    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-workload-socket\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:32:39.616081 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.615550    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istiod-ca-cert\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:32:39.616081 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.615659    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eff0e18d-b03a-446a-b7c0-b294d6c84d50" (UID: "eff0e18d-b03a-446a-b7c0-b294d6c84d50"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:32:39.616081 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.615693    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-data" (OuterVolumeSpecName: "istio-data") pod "eff0e18d-b03a-446a-b7c0-b294d6c84d50" (UID: "eff0e18d-b03a-446a-b7c0-b294d6c84d50"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:32:39.617084 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.617053    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eff0e18d-b03a-446a-b7c0-b294d6c84d50" (UID: "eff0e18d-b03a-446a-b7c0-b294d6c84d50"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 22:32:39.617311 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.617286    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eff0e18d-b03a-446a-b7c0-b294d6c84d50" (UID: "eff0e18d-b03a-446a-b7c0-b294d6c84d50"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:32:39.617311 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.617297    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eff0e18d-b03a-446a-b7c0-b294d6c84d50-kube-api-access-5rv7q" (OuterVolumeSpecName: "kube-api-access-5rv7q") pod "eff0e18d-b03a-446a-b7c0-b294d6c84d50" (UID: "eff0e18d-b03a-446a-b7c0-b294d6c84d50"). InnerVolumeSpecName "kube-api-access-5rv7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:32:39.617434 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.617314    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-token" (OuterVolumeSpecName: "istio-token") pod "eff0e18d-b03a-446a-b7c0-b294d6c84d50" (UID: "eff0e18d-b03a-446a-b7c0-b294d6c84d50"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:32:39.717037 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.717003    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-data\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:32:39.717037 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.717030    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-podinfo\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:32:39.717037 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.717039    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-envoy\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:32:39.717313 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.717050    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rv7q\" (UniqueName: \"kubernetes.io/projected/eff0e18d-b03a-446a-b7c0-b294d6c84d50-kube-api-access-5rv7q\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:32:39.717313 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.717080    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff0e18d-b03a-446a-b7c0-b294d6c84d50-credential-socket\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:32:39.717313 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:39.717091    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff0e18d-b03a-446a-b7c0-b294d6c84d50-istio-token\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:32:40.146501 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:40.146470    2574 generic.go:356] "Generic (PLEG): container finished" podID="eff0e18d-b03a-446a-b7c0-b294d6c84d50" containerID="265b5549b1d64259a503699ce8bd167bb3176bf9b7ba994250f6f3caa74826c2" exitCode=0
Jul 28 22:32:40.146708 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:40.146522    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"
Jul 28 22:32:40.146708 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:40.146551    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv" event={"ID":"eff0e18d-b03a-446a-b7c0-b294d6c84d50","Type":"ContainerDied","Data":"265b5549b1d64259a503699ce8bd167bb3176bf9b7ba994250f6f3caa74826c2"}
Jul 28 22:32:40.146708 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:40.146607    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv" event={"ID":"eff0e18d-b03a-446a-b7c0-b294d6c84d50","Type":"ContainerDied","Data":"e09052ec1b90ee202028e02db773026be0907082439e69277d18153dd5cac2a3"}
Jul 28 22:32:40.146708 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:40.146631    2574 scope.go:117] "RemoveContainer" containerID="265b5549b1d64259a503699ce8bd167bb3176bf9b7ba994250f6f3caa74826c2"
Jul 28 22:32:40.156316 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:40.156295    2574 scope.go:117] "RemoveContainer" containerID="ec2d978f648adea8e67a5bc739419f6c6bb7d8172f7919f544b77faba5350b58"
Jul 28 22:32:40.164948 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:40.164922    2574 scope.go:117] "RemoveContainer" containerID="265b5549b1d64259a503699ce8bd167bb3176bf9b7ba994250f6f3caa74826c2"
Jul 28 22:32:40.165521 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:32:40.165332    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"265b5549b1d64259a503699ce8bd167bb3176bf9b7ba994250f6f3caa74826c2\": container with ID starting with 265b5549b1d64259a503699ce8bd167bb3176bf9b7ba994250f6f3caa74826c2 not found: ID does not exist" containerID="265b5549b1d64259a503699ce8bd167bb3176bf9b7ba994250f6f3caa74826c2"
Jul 28 22:32:40.165521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:40.165374    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"265b5549b1d64259a503699ce8bd167bb3176bf9b7ba994250f6f3caa74826c2"} err="failed to get container status \"265b5549b1d64259a503699ce8bd167bb3176bf9b7ba994250f6f3caa74826c2\": rpc error: code = NotFound desc = could not find container \"265b5549b1d64259a503699ce8bd167bb3176bf9b7ba994250f6f3caa74826c2\": container with ID starting with 265b5549b1d64259a503699ce8bd167bb3176bf9b7ba994250f6f3caa74826c2 not found: ID does not exist"
Jul 28 22:32:40.165521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:40.165400    2574 scope.go:117] "RemoveContainer" containerID="ec2d978f648adea8e67a5bc739419f6c6bb7d8172f7919f544b77faba5350b58"
Jul 28 22:32:40.165740 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:32:40.165663    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec2d978f648adea8e67a5bc739419f6c6bb7d8172f7919f544b77faba5350b58\": container with ID starting with ec2d978f648adea8e67a5bc739419f6c6bb7d8172f7919f544b77faba5350b58 not found: ID does not exist" containerID="ec2d978f648adea8e67a5bc739419f6c6bb7d8172f7919f544b77faba5350b58"
Jul 28 22:32:40.165740 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:40.165688    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec2d978f648adea8e67a5bc739419f6c6bb7d8172f7919f544b77faba5350b58"} err="failed to get container status \"ec2d978f648adea8e67a5bc739419f6c6bb7d8172f7919f544b77faba5350b58\": rpc error: code = NotFound desc = could not find container \"ec2d978f648adea8e67a5bc739419f6c6bb7d8172f7919f544b77faba5350b58\": container with ID starting with ec2d978f648adea8e67a5bc739419f6c6bb7d8172f7919f544b77faba5350b58 not found: ID does not exist"
Jul 28 22:32:40.167180 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:40.167154    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"]
Jul 28 22:32:40.171133 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:40.171113    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-09e3e515-openshift-default-5898559c6b-s7xlv"]
Jul 28 22:32:41.493605 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:41.493563    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eff0e18d-b03a-446a-b7c0-b294d6c84d50" path="/var/lib/kubelet/pods/eff0e18d-b03a-446a-b7c0-b294d6c84d50/volumes"
Jul 28 22:32:52.388420 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:52.388384    2574 patch_prober.go:28] interesting pod/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": read tcp 10.133.0.2:58278->10.133.0.48:15021: read: connection reset by peer" start-of-body=
Jul 28 22:32:52.388983 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:52.388455    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh" podUID="58ac102e-4961-4250-a49f-bafec0dc3e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": read tcp 10.133.0.2:58278->10.133.0.48:15021: read: connection reset by peer"
Jul 28 22:32:53.209624 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:53.209549    2574 generic.go:356] "Generic (PLEG): container finished" podID="58ac102e-4961-4250-a49f-bafec0dc3e3d" containerID="952efef98968b934cbff69e9edb4385caa27f844a650cfc612b09b5d932efc9c" exitCode=0
Jul 28 22:32:53.209785 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:53.209624    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh" event={"ID":"58ac102e-4961-4250-a49f-bafec0dc3e3d","Type":"ContainerDied","Data":"952efef98968b934cbff69e9edb4385caa27f844a650cfc612b09b5d932efc9c"}
Jul 28 22:32:53.210022 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:53.210008    2574 scope.go:117] "RemoveContainer" containerID="952efef98968b934cbff69e9edb4385caa27f844a650cfc612b09b5d932efc9c"
Jul 28 22:32:53.210960 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:53.210935    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:32:53.211055 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:53.210999    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:32:53.211055 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:53.211033    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:32:54.217040 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:54.217007    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh" event={"ID":"58ac102e-4961-4250-a49f-bafec0dc3e3d","Type":"ContainerStarted","Data":"2855c85020e57ccc1d8309ae9e420ad530d011cec86f989ab86f0bf670484ead"}
Jul 28 22:32:54.577438 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:54.577357    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:54.577607 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:54.577587    2574 patch_prober.go:28] interesting pod/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 28 22:32:54.577652 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:54.577632    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh" podUID="58ac102e-4961-4250-a49f-bafec0dc3e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 28 22:32:55.577821 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:55.577786    2574 patch_prober.go:28] interesting pod/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 28 22:32:55.578223 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:55.577841    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh" podUID="58ac102e-4961-4250-a49f-bafec0dc3e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 28 22:32:56.578409 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:56.578369    2574 patch_prober.go:28] interesting pod/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 28 22:32:56.578808 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:56.578428    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh" podUID="58ac102e-4961-4250-a49f-bafec0dc3e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 28 22:32:57.578066 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:57.578027    2574 patch_prober.go:28] interesting pod/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 28 22:32:57.578284 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:57.578082    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh" podUID="58ac102e-4961-4250-a49f-bafec0dc3e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 28 22:32:58.581414 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:58.581378    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:58.581829 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:58.581722    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:32:58.582401 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:32:58.582384    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:33:09.460704 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:09.460668    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"]
Jul 28 22:33:09.461362 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:09.460918    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh" podUID="58ac102e-4961-4250-a49f-bafec0dc3e3d" containerName="istio-proxy" containerID="cri-o://2855c85020e57ccc1d8309ae9e420ad530d011cec86f989ab86f0bf670484ead" gracePeriod=30
Jul 28 22:33:10.234740 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.234707    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"]
Jul 28 22:33:10.235273 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.235251    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eff0e18d-b03a-446a-b7c0-b294d6c84d50" containerName="istio-proxy"
Jul 28 22:33:10.235364 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.235273    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="eff0e18d-b03a-446a-b7c0-b294d6c84d50" containerName="istio-proxy"
Jul 28 22:33:10.235364 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.235306    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eff0e18d-b03a-446a-b7c0-b294d6c84d50" containerName="istio-proxy"
Jul 28 22:33:10.235364 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.235315    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="eff0e18d-b03a-446a-b7c0-b294d6c84d50" containerName="istio-proxy"
Jul 28 22:33:10.235589 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.235404    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="eff0e18d-b03a-446a-b7c0-b294d6c84d50" containerName="istio-proxy"
Jul 28 22:33:10.238360 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.238337    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.240517 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.240492    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e1a8249b-openshift-default-dockercfg-8zs8s\""
Jul 28 22:33:10.250706 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.250678    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"]
Jul 28 22:33:10.299872 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.294450    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-envoy\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.299872 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.294513    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-workload-socket\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.299872 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.294582    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdrg7\" (UniqueName: \"kubernetes.io/projected/a3b468e0-6be5-4c0b-969e-802c9a766794-kube-api-access-mdrg7\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.299872 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.294617    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-token\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.299872 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.294670    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-credential-socket\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.299872 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.294698    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b468e0-6be5-4c0b-969e-802c9a766794-istiod-ca-cert\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.299872 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.294742    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-workload-certs\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.299872 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.294777    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-data\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.299872 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.294800    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-podinfo\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.395686 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.395647    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdrg7\" (UniqueName: \"kubernetes.io/projected/a3b468e0-6be5-4c0b-969e-802c9a766794-kube-api-access-mdrg7\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.395686 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.395689    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-token\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.395946 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.395730    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-credential-socket\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.395946 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.395746    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b468e0-6be5-4c0b-969e-802c9a766794-istiod-ca-cert\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.395946 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.395771    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-workload-certs\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.395946 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.395789    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-data\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.395946 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.395805    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-podinfo\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.395946 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.395898    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-envoy\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.395946 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.395945    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-workload-socket\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.396333 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.396218    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-credential-socket\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.396333 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.396290    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-workload-certs\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.396444 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.396379    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-data\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.396501 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.396449    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-workload-socket\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.396686 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.396663    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b468e0-6be5-4c0b-969e-802c9a766794-istiod-ca-cert\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.398140 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.398120    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-envoy\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.398424 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.398407    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-podinfo\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.403211 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.403183    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-token\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.403872 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.403850    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdrg7\" (UniqueName: \"kubernetes.io/projected/a3b468e0-6be5-4c0b-969e-802c9a766794-kube-api-access-mdrg7\") pod \"e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") " pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.549817 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.549715    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:10.691026 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.690993    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"]
Jul 28 22:33:10.694650 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:33:10.694620    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3b468e0_6be5_4c0b_969e_802c9a766794.slice/crio-77e98763f5760e0b33a00ab9a511ca2b44617c98137e214943105ceb33ee30d8 WatchSource:0}: Error finding container 77e98763f5760e0b33a00ab9a511ca2b44617c98137e214943105ceb33ee30d8: Status 404 returned error can't find the container with id 77e98763f5760e0b33a00ab9a511ca2b44617c98137e214943105ceb33ee30d8
Jul 28 22:33:10.696893 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.696857    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:33:10.696998 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.696927    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:33:10.696998 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:10.696962    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:33:11.309120 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:11.309082    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5" event={"ID":"a3b468e0-6be5-4c0b-969e-802c9a766794","Type":"ContainerStarted","Data":"f0d60d9a94e25a9573df96fc04116adddc59ee0c72dd6dfbc4b9f60d4a94d0ac"}
Jul 28 22:33:11.309347 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:11.309136    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5" event={"ID":"a3b468e0-6be5-4c0b-969e-802c9a766794","Type":"ContainerStarted","Data":"77e98763f5760e0b33a00ab9a511ca2b44617c98137e214943105ceb33ee30d8"}
Jul 28 22:33:11.344336 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:11.344280    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5" podStartSLOduration=1.3442626500000001 podStartE2EDuration="1.34426265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:33:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:33:11.343613278 +0000 UTC m=+1720.327755894" watchObservedRunningTime="2026-07-28 22:33:11.34426265 +0000 UTC m=+1720.328405270"
Jul 28 22:33:11.550901 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:11.550856    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:11.556000 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:11.555972    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:12.314578 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:12.314544    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:12.315639 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:12.315618    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:13.584492 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:13.584457    2574 patch_prober.go:28] interesting pod/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 22:33:13.584938 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:13.584535    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh" podUID="58ac102e-4961-4250-a49f-bafec0dc3e3d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 22:33:14.732061 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.731975    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:33:14.833634 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.833605    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-data\") pod \"58ac102e-4961-4250-a49f-bafec0dc3e3d\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") "
Jul 28 22:33:14.833821 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.833653    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-workload-socket\") pod \"58ac102e-4961-4250-a49f-bafec0dc3e3d\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") "
Jul 28 22:33:14.833821 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.833702    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-envoy\") pod \"58ac102e-4961-4250-a49f-bafec0dc3e3d\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") "
Jul 28 22:33:14.833821 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.833721    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-podinfo\") pod \"58ac102e-4961-4250-a49f-bafec0dc3e3d\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") "
Jul 28 22:33:14.833821 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.833737    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-workload-certs\") pod \"58ac102e-4961-4250-a49f-bafec0dc3e3d\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") "
Jul 28 22:33:14.833821 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.833787    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-credential-socket\") pod \"58ac102e-4961-4250-a49f-bafec0dc3e3d\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") "
Jul 28 22:33:14.833821 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.833813    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dt6k7\" (UniqueName: \"kubernetes.io/projected/58ac102e-4961-4250-a49f-bafec0dc3e3d-kube-api-access-dt6k7\") pod \"58ac102e-4961-4250-a49f-bafec0dc3e3d\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") "
Jul 28 22:33:14.834141 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.833848    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58ac102e-4961-4250-a49f-bafec0dc3e3d-istiod-ca-cert\") pod \"58ac102e-4961-4250-a49f-bafec0dc3e3d\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") "
Jul 28 22:33:14.834141 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.833884    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-token\") pod \"58ac102e-4961-4250-a49f-bafec0dc3e3d\" (UID: \"58ac102e-4961-4250-a49f-bafec0dc3e3d\") "
Jul 28 22:33:14.834141 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.834045    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "58ac102e-4961-4250-a49f-bafec0dc3e3d" (UID: "58ac102e-4961-4250-a49f-bafec0dc3e3d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:33:14.834141 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.834119    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "58ac102e-4961-4250-a49f-bafec0dc3e3d" (UID: "58ac102e-4961-4250-a49f-bafec0dc3e3d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:33:14.834338 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.834276    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-credential-socket\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:33:14.834338 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.834295    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-workload-certs\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:33:14.834449 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.834412    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58ac102e-4961-4250-a49f-bafec0dc3e3d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "58ac102e-4961-4250-a49f-bafec0dc3e3d" (UID: "58ac102e-4961-4250-a49f-bafec0dc3e3d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:33:14.834506 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.834463    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "58ac102e-4961-4250-a49f-bafec0dc3e3d" (UID: "58ac102e-4961-4250-a49f-bafec0dc3e3d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:33:14.834586 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.834567    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-data" (OuterVolumeSpecName: "istio-data") pod "58ac102e-4961-4250-a49f-bafec0dc3e3d" (UID: "58ac102e-4961-4250-a49f-bafec0dc3e3d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:33:14.836547 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.836507    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58ac102e-4961-4250-a49f-bafec0dc3e3d-kube-api-access-dt6k7" (OuterVolumeSpecName: "kube-api-access-dt6k7") pod "58ac102e-4961-4250-a49f-bafec0dc3e3d" (UID: "58ac102e-4961-4250-a49f-bafec0dc3e3d"). InnerVolumeSpecName "kube-api-access-dt6k7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:33:14.836547 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.836535    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "58ac102e-4961-4250-a49f-bafec0dc3e3d" (UID: "58ac102e-4961-4250-a49f-bafec0dc3e3d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:33:14.836703 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.836586    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "58ac102e-4961-4250-a49f-bafec0dc3e3d" (UID: "58ac102e-4961-4250-a49f-bafec0dc3e3d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 22:33:14.836942 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.836922    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-token" (OuterVolumeSpecName: "istio-token") pod "58ac102e-4961-4250-a49f-bafec0dc3e3d" (UID: "58ac102e-4961-4250-a49f-bafec0dc3e3d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:33:14.935623 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.935578    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-workload-socket\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:33:14.935623 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.935611    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-envoy\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:33:14.935623 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.935626    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-podinfo\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:33:14.935881 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.935637    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dt6k7\" (UniqueName: \"kubernetes.io/projected/58ac102e-4961-4250-a49f-bafec0dc3e3d-kube-api-access-dt6k7\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:33:14.935881 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.935649    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58ac102e-4961-4250-a49f-bafec0dc3e3d-istiod-ca-cert\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:33:14.935881 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.935660    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-token\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:33:14.935881 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:14.935670    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58ac102e-4961-4250-a49f-bafec0dc3e3d-istio-data\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:33:15.330023 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:15.329993    2574 generic.go:356] "Generic (PLEG): container finished" podID="58ac102e-4961-4250-a49f-bafec0dc3e3d" containerID="2855c85020e57ccc1d8309ae9e420ad530d011cec86f989ab86f0bf670484ead" exitCode=0
Jul 28 22:33:15.330213 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:15.330038    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh" event={"ID":"58ac102e-4961-4250-a49f-bafec0dc3e3d","Type":"ContainerDied","Data":"2855c85020e57ccc1d8309ae9e420ad530d011cec86f989ab86f0bf670484ead"}
Jul 28 22:33:15.330213 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:15.330048    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"
Jul 28 22:33:15.330213 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:15.330111    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh" event={"ID":"58ac102e-4961-4250-a49f-bafec0dc3e3d","Type":"ContainerDied","Data":"8c1d43be1b859b11f1b1958e1f7fcde48667fbdf0206a64313015332fdcc8377"}
Jul 28 22:33:15.330213 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:15.330135    2574 scope.go:117] "RemoveContainer" containerID="2855c85020e57ccc1d8309ae9e420ad530d011cec86f989ab86f0bf670484ead"
Jul 28 22:33:15.339371 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:15.339349    2574 scope.go:117] "RemoveContainer" containerID="952efef98968b934cbff69e9edb4385caa27f844a650cfc612b09b5d932efc9c"
Jul 28 22:33:15.348350 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:15.348332    2574 scope.go:117] "RemoveContainer" containerID="2855c85020e57ccc1d8309ae9e420ad530d011cec86f989ab86f0bf670484ead"
Jul 28 22:33:15.348696 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:33:15.348664    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2855c85020e57ccc1d8309ae9e420ad530d011cec86f989ab86f0bf670484ead\": container with ID starting with 2855c85020e57ccc1d8309ae9e420ad530d011cec86f989ab86f0bf670484ead not found: ID does not exist" containerID="2855c85020e57ccc1d8309ae9e420ad530d011cec86f989ab86f0bf670484ead"
Jul 28 22:33:15.348785 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:15.348700    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2855c85020e57ccc1d8309ae9e420ad530d011cec86f989ab86f0bf670484ead"} err="failed to get container status \"2855c85020e57ccc1d8309ae9e420ad530d011cec86f989ab86f0bf670484ead\": rpc error: code = NotFound desc = could not find container \"2855c85020e57ccc1d8309ae9e420ad530d011cec86f989ab86f0bf670484ead\": container with ID starting with 2855c85020e57ccc1d8309ae9e420ad530d011cec86f989ab86f0bf670484ead not found: ID does not exist"
Jul 28 22:33:15.348785 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:15.348720    2574 scope.go:117] "RemoveContainer" containerID="952efef98968b934cbff69e9edb4385caa27f844a650cfc612b09b5d932efc9c"
Jul 28 22:33:15.349064 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:33:15.349029    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"952efef98968b934cbff69e9edb4385caa27f844a650cfc612b09b5d932efc9c\": container with ID starting with 952efef98968b934cbff69e9edb4385caa27f844a650cfc612b09b5d932efc9c not found: ID does not exist" containerID="952efef98968b934cbff69e9edb4385caa27f844a650cfc612b09b5d932efc9c"
Jul 28 22:33:15.349154 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:15.349071    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"952efef98968b934cbff69e9edb4385caa27f844a650cfc612b09b5d932efc9c"} err="failed to get container status \"952efef98968b934cbff69e9edb4385caa27f844a650cfc612b09b5d932efc9c\": rpc error: code = NotFound desc = could not find container \"952efef98968b934cbff69e9edb4385caa27f844a650cfc612b09b5d932efc9c\": container with ID starting with 952efef98968b934cbff69e9edb4385caa27f844a650cfc612b09b5d932efc9c not found: ID does not exist"
Jul 28 22:33:15.351498 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:15.351476    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"]
Jul 28 22:33:15.355344 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:15.355324    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-6a50116a-gw-openshift-default-7b5b959984-9f9gh"]
Jul 28 22:33:15.493701 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:15.493666    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58ac102e-4961-4250-a49f-bafec0dc3e3d" path="/var/lib/kubelet/pods/58ac102e-4961-4250-a49f-bafec0dc3e3d/volumes"
Jul 28 22:33:27.506548 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:27.506506    2574 patch_prober.go:28] interesting pod/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": read tcp 10.133.0.2:55808->10.133.0.49:15021: read: connection reset by peer" start-of-body=
Jul 28 22:33:27.507119 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:27.506577    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5" podUID="a3b468e0-6be5-4c0b-969e-802c9a766794" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": read tcp 10.133.0.2:55808->10.133.0.49:15021: read: connection reset by peer"
Jul 28 22:33:28.409622 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:28.409586    2574 generic.go:356] "Generic (PLEG): container finished" podID="a3b468e0-6be5-4c0b-969e-802c9a766794" containerID="f0d60d9a94e25a9573df96fc04116adddc59ee0c72dd6dfbc4b9f60d4a94d0ac" exitCode=0
Jul 28 22:33:28.409812 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:28.409663    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5" event={"ID":"a3b468e0-6be5-4c0b-969e-802c9a766794","Type":"ContainerDied","Data":"f0d60d9a94e25a9573df96fc04116adddc59ee0c72dd6dfbc4b9f60d4a94d0ac"}
Jul 28 22:33:28.410123 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:28.410107    2574 scope.go:117] "RemoveContainer" containerID="f0d60d9a94e25a9573df96fc04116adddc59ee0c72dd6dfbc4b9f60d4a94d0ac"
Jul 28 22:33:28.411132 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:28.411105    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:33:28.411201 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:28.411171    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:33:28.411255 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:28.411206    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:33:29.416761 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:29.416728    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5" event={"ID":"a3b468e0-6be5-4c0b-969e-802c9a766794","Type":"ContainerStarted","Data":"c47f5c4803cd4801c9a5d635b8d150f5846d4d7f30775b5b7e99a241f8ee2e5a"}
Jul 28 22:33:29.550515 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:29.550479    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:29.550914 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:29.550821    2574 patch_prober.go:28] interesting pod/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 28 22:33:29.551084 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:29.551062    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5" podUID="a3b468e0-6be5-4c0b-969e-802c9a766794" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 28 22:33:30.550354 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:30.550312    2574 patch_prober.go:28] interesting pod/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 28 22:33:30.550863 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:30.550429    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5" podUID="a3b468e0-6be5-4c0b-969e-802c9a766794" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 28 22:33:31.551212 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:31.551178    2574 patch_prober.go:28] interesting pod/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 28 22:33:31.552211 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:31.551773    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5" podUID="a3b468e0-6be5-4c0b-969e-802c9a766794" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 28 22:33:32.550785 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:32.550743    2574 patch_prober.go:28] interesting pod/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 28 22:33:32.551126 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:32.551092    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5" podUID="a3b468e0-6be5-4c0b-969e-802c9a766794" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 28 22:33:33.553926 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:33.553895    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:33.554336 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:33.554178    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:33.554902 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:33.554885    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:44.887574 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:44.887531    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"]
Jul 28 22:33:44.888196 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:44.887740    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5" podUID="a3b468e0-6be5-4c0b-969e-802c9a766794" containerName="istio-proxy" containerID="cri-o://c47f5c4803cd4801c9a5d635b8d150f5846d4d7f30775b5b7e99a241f8ee2e5a" gracePeriod=30
Jul 28 22:33:45.659430 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.659399    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"]
Jul 28 22:33:45.659960 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.659943    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58ac102e-4961-4250-a49f-bafec0dc3e3d" containerName="istio-proxy"
Jul 28 22:33:45.660006 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.659963    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="58ac102e-4961-4250-a49f-bafec0dc3e3d" containerName="istio-proxy"
Jul 28 22:33:45.660050 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.660021    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="eff0e18d-b03a-446a-b7c0-b294d6c84d50" containerName="istio-proxy"
Jul 28 22:33:45.660050 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.660030    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="58ac102e-4961-4250-a49f-bafec0dc3e3d" containerName="istio-proxy"
Jul 28 22:33:45.660050 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.660039    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="58ac102e-4961-4250-a49f-bafec0dc3e3d" containerName="istio-proxy"
Jul 28 22:33:45.662496 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.662478    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.664937 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.664912    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-f8361843-openshift-default-dockercfg-t9v4z\""
Jul 28 22:33:45.675835 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.675809    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"]
Jul 28 22:33:45.829011 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.828975    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93062a00-8491-4099-9c55-1d1b64252939-istiod-ca-cert\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.829194 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.829036    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-credential-socket\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.829194 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.829125    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-workload-socket\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.829194 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.829170    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-workload-certs\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.829384 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.829193    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93062a00-8491-4099-9c55-1d1b64252939-istio-podinfo\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.829384 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.829219    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-istio-envoy\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.829384 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.829282    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-istio-data\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.829384 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.829362    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j2zh\" (UniqueName: \"kubernetes.io/projected/93062a00-8491-4099-9c55-1d1b64252939-kube-api-access-5j2zh\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.829549 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.829392    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93062a00-8491-4099-9c55-1d1b64252939-istio-token\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.930023 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.929923    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93062a00-8491-4099-9c55-1d1b64252939-istio-token\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.930023 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.929984    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93062a00-8491-4099-9c55-1d1b64252939-istiod-ca-cert\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.930023 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.930017    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-credential-socket\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.930600 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.930051    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-workload-socket\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.930600 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.930252    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-workload-certs\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.930600 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.930284    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93062a00-8491-4099-9c55-1d1b64252939-istio-podinfo\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.930600 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.930328    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-istio-envoy\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.930600 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.930354    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-istio-data\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.930600 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.930460    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-credential-socket\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.930600 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.930507    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-workload-socket\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.930600 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.930452    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j2zh\" (UniqueName: \"kubernetes.io/projected/93062a00-8491-4099-9c55-1d1b64252939-kube-api-access-5j2zh\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.930600 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.930594    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-workload-certs\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.930946 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.930732    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-istio-data\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.930985 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.930945    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93062a00-8491-4099-9c55-1d1b64252939-istiod-ca-cert\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.932609 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.932590    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-istio-envoy\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.932837 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.932816    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93062a00-8491-4099-9c55-1d1b64252939-istio-podinfo\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.938147 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.938117    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j2zh\" (UniqueName: \"kubernetes.io/projected/93062a00-8491-4099-9c55-1d1b64252939-kube-api-access-5j2zh\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.938325 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.938171    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93062a00-8491-4099-9c55-1d1b64252939-istio-token\") pod \"e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") " pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:45.972503 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:45.972474    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:46.119407 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:46.119383    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"]
Jul 28 22:33:46.121560 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:33:46.121521    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93062a00_8491_4099_9c55_1d1b64252939.slice/crio-bc4b47062f235072728cf87d9c34a86a5c5e56c6bb3658104886ebbd4d0e21a8 WatchSource:0}: Error finding container bc4b47062f235072728cf87d9c34a86a5c5e56c6bb3658104886ebbd4d0e21a8: Status 404 returned error can't find the container with id bc4b47062f235072728cf87d9c34a86a5c5e56c6bb3658104886ebbd4d0e21a8
Jul 28 22:33:46.123809 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:46.123777    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:33:46.123893 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:46.123843    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:33:46.123893 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:46.123873    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:33:46.519758 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:46.519720    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9" event={"ID":"93062a00-8491-4099-9c55-1d1b64252939","Type":"ContainerStarted","Data":"674c2786b10331adfecfaa58a03454a78ee7e5e5137bcd7bac0ceb6eb40e259d"}
Jul 28 22:33:46.519935 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:46.519773    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9" event={"ID":"93062a00-8491-4099-9c55-1d1b64252939","Type":"ContainerStarted","Data":"bc4b47062f235072728cf87d9c34a86a5c5e56c6bb3658104886ebbd4d0e21a8"}
Jul 28 22:33:46.541708 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:46.541650    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9" podStartSLOduration=1.54162865 podStartE2EDuration="1.54162865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:33:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:33:46.538451454 +0000 UTC m=+1755.522594077" watchObservedRunningTime="2026-07-28 22:33:46.54162865 +0000 UTC m=+1755.525771272"
Jul 28 22:33:46.972780 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:46.972751    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:47.979338 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:47.979256    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:48.529850 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:48.529812    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:48.531014 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:48.530989    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:33:48.556718 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:48.556688    2574 patch_prober.go:28] interesting pod/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 22:33:48.556881 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:48.556772    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5" podUID="a3b468e0-6be5-4c0b-969e-802c9a766794" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 22:33:50.141371 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.141344    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:50.173560 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.173480    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-envoy\") pod \"a3b468e0-6be5-4c0b-969e-802c9a766794\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") "
Jul 28 22:33:50.173560 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.173518    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdrg7\" (UniqueName: \"kubernetes.io/projected/a3b468e0-6be5-4c0b-969e-802c9a766794-kube-api-access-mdrg7\") pod \"a3b468e0-6be5-4c0b-969e-802c9a766794\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") "
Jul 28 22:33:50.173560 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.173538    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-workload-socket\") pod \"a3b468e0-6be5-4c0b-969e-802c9a766794\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") "
Jul 28 22:33:50.173821 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.173586    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b468e0-6be5-4c0b-969e-802c9a766794-istiod-ca-cert\") pod \"a3b468e0-6be5-4c0b-969e-802c9a766794\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") "
Jul 28 22:33:50.173821 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.173630    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-credential-socket\") pod \"a3b468e0-6be5-4c0b-969e-802c9a766794\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") "
Jul 28 22:33:50.173821 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.173652    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-token\") pod \"a3b468e0-6be5-4c0b-969e-802c9a766794\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") "
Jul 28 22:33:50.173821 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.173685    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-workload-certs\") pod \"a3b468e0-6be5-4c0b-969e-802c9a766794\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") "
Jul 28 22:33:50.173821 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.173699    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-data\") pod \"a3b468e0-6be5-4c0b-969e-802c9a766794\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") "
Jul 28 22:33:50.173821 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.173725    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-podinfo\") pod \"a3b468e0-6be5-4c0b-969e-802c9a766794\" (UID: \"a3b468e0-6be5-4c0b-969e-802c9a766794\") "
Jul 28 22:33:50.174121 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.173937    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a3b468e0-6be5-4c0b-969e-802c9a766794" (UID: "a3b468e0-6be5-4c0b-969e-802c9a766794"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:33:50.174121 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.174021    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3b468e0-6be5-4c0b-969e-802c9a766794-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a3b468e0-6be5-4c0b-969e-802c9a766794" (UID: "a3b468e0-6be5-4c0b-969e-802c9a766794"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:33:50.174249 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.174117    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-workload-socket\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:33:50.174249 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.174131    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a3b468e0-6be5-4c0b-969e-802c9a766794" (UID: "a3b468e0-6be5-4c0b-969e-802c9a766794"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:33:50.174368 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.174329    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a3b468e0-6be5-4c0b-969e-802c9a766794" (UID: "a3b468e0-6be5-4c0b-969e-802c9a766794"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:33:50.175117 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.175082    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-data" (OuterVolumeSpecName: "istio-data") pod "a3b468e0-6be5-4c0b-969e-802c9a766794" (UID: "a3b468e0-6be5-4c0b-969e-802c9a766794"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:33:50.176214 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.176185    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a3b468e0-6be5-4c0b-969e-802c9a766794" (UID: "a3b468e0-6be5-4c0b-969e-802c9a766794"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 22:33:50.176214 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.176202    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-token" (OuterVolumeSpecName: "istio-token") pod "a3b468e0-6be5-4c0b-969e-802c9a766794" (UID: "a3b468e0-6be5-4c0b-969e-802c9a766794"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:33:50.176401 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.176298    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a3b468e0-6be5-4c0b-969e-802c9a766794" (UID: "a3b468e0-6be5-4c0b-969e-802c9a766794"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:33:50.176401 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.176322    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3b468e0-6be5-4c0b-969e-802c9a766794-kube-api-access-mdrg7" (OuterVolumeSpecName: "kube-api-access-mdrg7") pod "a3b468e0-6be5-4c0b-969e-802c9a766794" (UID: "a3b468e0-6be5-4c0b-969e-802c9a766794"). InnerVolumeSpecName "kube-api-access-mdrg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:33:50.275220 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.275184    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-credential-socket\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:33:50.275220 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.275214    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-token\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:33:50.275220 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.275225    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-workload-certs\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:33:50.275220 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.275245    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-data\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:33:50.275554 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.275253    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-podinfo\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:33:50.275554 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.275260    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b468e0-6be5-4c0b-969e-802c9a766794-istio-envoy\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:33:50.275554 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.275268    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdrg7\" (UniqueName: \"kubernetes.io/projected/a3b468e0-6be5-4c0b-969e-802c9a766794-kube-api-access-mdrg7\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:33:50.275554 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.275276    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b468e0-6be5-4c0b-969e-802c9a766794-istiod-ca-cert\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:33:50.541318 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.541283    2574 generic.go:356] "Generic (PLEG): container finished" podID="a3b468e0-6be5-4c0b-969e-802c9a766794" containerID="c47f5c4803cd4801c9a5d635b8d150f5846d4d7f30775b5b7e99a241f8ee2e5a" exitCode=0
Jul 28 22:33:50.541519 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.541323    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"
Jul 28 22:33:50.541519 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.541369    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5" event={"ID":"a3b468e0-6be5-4c0b-969e-802c9a766794","Type":"ContainerDied","Data":"c47f5c4803cd4801c9a5d635b8d150f5846d4d7f30775b5b7e99a241f8ee2e5a"}
Jul 28 22:33:50.541519 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.541434    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5" event={"ID":"a3b468e0-6be5-4c0b-969e-802c9a766794","Type":"ContainerDied","Data":"77e98763f5760e0b33a00ab9a511ca2b44617c98137e214943105ceb33ee30d8"}
Jul 28 22:33:50.541519 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.541457    2574 scope.go:117] "RemoveContainer" containerID="c47f5c4803cd4801c9a5d635b8d150f5846d4d7f30775b5b7e99a241f8ee2e5a"
Jul 28 22:33:50.551216 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.551190    2574 scope.go:117] "RemoveContainer" containerID="f0d60d9a94e25a9573df96fc04116adddc59ee0c72dd6dfbc4b9f60d4a94d0ac"
Jul 28 22:33:50.559472 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.559448    2574 scope.go:117] "RemoveContainer" containerID="c47f5c4803cd4801c9a5d635b8d150f5846d4d7f30775b5b7e99a241f8ee2e5a"
Jul 28 22:33:50.559784 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:33:50.559764    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c47f5c4803cd4801c9a5d635b8d150f5846d4d7f30775b5b7e99a241f8ee2e5a\": container with ID starting with c47f5c4803cd4801c9a5d635b8d150f5846d4d7f30775b5b7e99a241f8ee2e5a not found: ID does not exist" containerID="c47f5c4803cd4801c9a5d635b8d150f5846d4d7f30775b5b7e99a241f8ee2e5a"
Jul 28 22:33:50.559851 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.559795    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c47f5c4803cd4801c9a5d635b8d150f5846d4d7f30775b5b7e99a241f8ee2e5a"} err="failed to get container status \"c47f5c4803cd4801c9a5d635b8d150f5846d4d7f30775b5b7e99a241f8ee2e5a\": rpc error: code = NotFound desc = could not find container \"c47f5c4803cd4801c9a5d635b8d150f5846d4d7f30775b5b7e99a241f8ee2e5a\": container with ID starting with c47f5c4803cd4801c9a5d635b8d150f5846d4d7f30775b5b7e99a241f8ee2e5a not found: ID does not exist"
Jul 28 22:33:50.559851 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.559812    2574 scope.go:117] "RemoveContainer" containerID="f0d60d9a94e25a9573df96fc04116adddc59ee0c72dd6dfbc4b9f60d4a94d0ac"
Jul 28 22:33:50.560090 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:33:50.560066    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0d60d9a94e25a9573df96fc04116adddc59ee0c72dd6dfbc4b9f60d4a94d0ac\": container with ID starting with f0d60d9a94e25a9573df96fc04116adddc59ee0c72dd6dfbc4b9f60d4a94d0ac not found: ID does not exist" containerID="f0d60d9a94e25a9573df96fc04116adddc59ee0c72dd6dfbc4b9f60d4a94d0ac"
Jul 28 22:33:50.560153 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.560094    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0d60d9a94e25a9573df96fc04116adddc59ee0c72dd6dfbc4b9f60d4a94d0ac"} err="failed to get container status \"f0d60d9a94e25a9573df96fc04116adddc59ee0c72dd6dfbc4b9f60d4a94d0ac\": rpc error: code = NotFound desc = could not find container \"f0d60d9a94e25a9573df96fc04116adddc59ee0c72dd6dfbc4b9f60d4a94d0ac\": container with ID starting with f0d60d9a94e25a9573df96fc04116adddc59ee0c72dd6dfbc4b9f60d4a94d0ac not found: ID does not exist"
Jul 28 22:33:50.565780 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.565756    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"]
Jul 28 22:33:50.577586 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:50.577552    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e1a8249b-openshift-default-674dfdfc56-mpnx5"]
Jul 28 22:33:51.494255 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:33:51.494194    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3b468e0-6be5-4c0b-969e-802c9a766794" path="/var/lib/kubelet/pods/a3b468e0-6be5-4c0b-969e-802c9a766794/volumes"
Jul 28 22:34:02.603756 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:02.603715    2574 generic.go:356] "Generic (PLEG): container finished" podID="93062a00-8491-4099-9c55-1d1b64252939" containerID="674c2786b10331adfecfaa58a03454a78ee7e5e5137bcd7bac0ceb6eb40e259d" exitCode=0
Jul 28 22:34:02.604259 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:02.603786    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9" event={"ID":"93062a00-8491-4099-9c55-1d1b64252939","Type":"ContainerDied","Data":"674c2786b10331adfecfaa58a03454a78ee7e5e5137bcd7bac0ceb6eb40e259d"}
Jul 28 22:34:02.604259 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:02.604185    2574 scope.go:117] "RemoveContainer" containerID="674c2786b10331adfecfaa58a03454a78ee7e5e5137bcd7bac0ceb6eb40e259d"
Jul 28 22:34:02.610832 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:02.605921    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:34:02.610832 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:02.606093    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:34:02.610832 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:02.606304    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:34:03.529911 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:03.529863    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:34:03.610861 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:03.610830    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9" event={"ID":"93062a00-8491-4099-9c55-1d1b64252939","Type":"ContainerStarted","Data":"63d4eb27736145f7904f2e40c8097cf8a628b27eaf1d6db8b8431222c674e54f"}
Jul 28 22:34:03.973515 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:03.973465    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:34:03.974720 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:03.974692    2574 patch_prober.go:28] interesting pod/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 28 22:34:03.974854 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:03.974750    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9" podUID="93062a00-8491-4099-9c55-1d1b64252939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 28 22:34:04.973330 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:04.973297    2574 patch_prober.go:28] interesting pod/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 28 22:34:04.973735 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:04.973352    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9" podUID="93062a00-8491-4099-9c55-1d1b64252939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 28 22:34:05.973435 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:05.973389    2574 patch_prober.go:28] interesting pod/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 28 22:34:05.973931 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:05.973481    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9" podUID="93062a00-8491-4099-9c55-1d1b64252939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 28 22:34:06.973537 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:06.973499    2574 patch_prober.go:28] interesting pod/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 28 22:34:06.973920 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:06.973556    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9" podUID="93062a00-8491-4099-9c55-1d1b64252939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 28 22:34:07.977519 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:07.977489    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:34:07.977894 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:07.977757    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:34:07.978427 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:07.978406    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:34:19.027539 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:19.027499    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"]
Jul 28 22:34:19.030616 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:19.030575    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9" podUID="93062a00-8491-4099-9c55-1d1b64252939" containerName="istio-proxy" containerID="cri-o://63d4eb27736145f7904f2e40c8097cf8a628b27eaf1d6db8b8431222c674e54f" gracePeriod=30
Jul 28 22:34:22.980297 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:22.980256    2574 patch_prober.go:28] interesting pod/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 28 22:34:22.980752 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:22.980320    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9" podUID="93062a00-8491-4099-9c55-1d1b64252939" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 22:34:24.301639 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.301572    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:34:24.487486 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.487446    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93062a00-8491-4099-9c55-1d1b64252939-istiod-ca-cert\") pod \"93062a00-8491-4099-9c55-1d1b64252939\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") "
Jul 28 22:34:24.487486 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.487486    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-credential-socket\") pod \"93062a00-8491-4099-9c55-1d1b64252939\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") "
Jul 28 22:34:24.487703 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.487516    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-istio-envoy\") pod \"93062a00-8491-4099-9c55-1d1b64252939\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") "
Jul 28 22:34:24.487703 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.487555    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-workload-certs\") pod \"93062a00-8491-4099-9c55-1d1b64252939\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") "
Jul 28 22:34:24.487703 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.487600    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-workload-socket\") pod \"93062a00-8491-4099-9c55-1d1b64252939\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") "
Jul 28 22:34:24.487703 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.487629    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93062a00-8491-4099-9c55-1d1b64252939-istio-podinfo\") pod \"93062a00-8491-4099-9c55-1d1b64252939\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") "
Jul 28 22:34:24.487901 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.487707    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93062a00-8491-4099-9c55-1d1b64252939-istio-token\") pod \"93062a00-8491-4099-9c55-1d1b64252939\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") "
Jul 28 22:34:24.487901 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.487742    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-istio-data\") pod \"93062a00-8491-4099-9c55-1d1b64252939\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") "
Jul 28 22:34:24.487901 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.487815    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "93062a00-8491-4099-9c55-1d1b64252939" (UID: "93062a00-8491-4099-9c55-1d1b64252939"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:34:24.487901 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.487856    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5j2zh\" (UniqueName: \"kubernetes.io/projected/93062a00-8491-4099-9c55-1d1b64252939-kube-api-access-5j2zh\") pod \"93062a00-8491-4099-9c55-1d1b64252939\" (UID: \"93062a00-8491-4099-9c55-1d1b64252939\") "
Jul 28 22:34:24.488109 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.487911    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "93062a00-8491-4099-9c55-1d1b64252939" (UID: "93062a00-8491-4099-9c55-1d1b64252939"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:34:24.488109 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.487986    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93062a00-8491-4099-9c55-1d1b64252939-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "93062a00-8491-4099-9c55-1d1b64252939" (UID: "93062a00-8491-4099-9c55-1d1b64252939"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:34:24.488222 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.488153    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "93062a00-8491-4099-9c55-1d1b64252939" (UID: "93062a00-8491-4099-9c55-1d1b64252939"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:34:24.488222 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.488166    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93062a00-8491-4099-9c55-1d1b64252939-istiod-ca-cert\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:34:24.488222 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.488184    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-credential-socket\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:34:24.488222 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.488197    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-workload-certs\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:34:24.488918 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.488890    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-istio-data" (OuterVolumeSpecName: "istio-data") pod "93062a00-8491-4099-9c55-1d1b64252939" (UID: "93062a00-8491-4099-9c55-1d1b64252939"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:34:24.490005 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.489969    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93062a00-8491-4099-9c55-1d1b64252939-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "93062a00-8491-4099-9c55-1d1b64252939" (UID: "93062a00-8491-4099-9c55-1d1b64252939"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 22:34:24.490110 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.489998    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93062a00-8491-4099-9c55-1d1b64252939-istio-token" (OuterVolumeSpecName: "istio-token") pod "93062a00-8491-4099-9c55-1d1b64252939" (UID: "93062a00-8491-4099-9c55-1d1b64252939"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:34:24.490110 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.490015    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93062a00-8491-4099-9c55-1d1b64252939-kube-api-access-5j2zh" (OuterVolumeSpecName: "kube-api-access-5j2zh") pod "93062a00-8491-4099-9c55-1d1b64252939" (UID: "93062a00-8491-4099-9c55-1d1b64252939"). InnerVolumeSpecName "kube-api-access-5j2zh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:34:24.490110 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.490064    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "93062a00-8491-4099-9c55-1d1b64252939" (UID: "93062a00-8491-4099-9c55-1d1b64252939"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:34:24.589483 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.589390    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5j2zh\" (UniqueName: \"kubernetes.io/projected/93062a00-8491-4099-9c55-1d1b64252939-kube-api-access-5j2zh\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:34:24.589483 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.589420    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-istio-envoy\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:34:24.589483 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.589430    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-workload-socket\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:34:24.589483 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.589440    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93062a00-8491-4099-9c55-1d1b64252939-istio-podinfo\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:34:24.589483 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.589448    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93062a00-8491-4099-9c55-1d1b64252939-istio-token\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:34:24.589483 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.589455    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93062a00-8491-4099-9c55-1d1b64252939-istio-data\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:34:24.715361 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.715325    2574 generic.go:356] "Generic (PLEG): container finished" podID="93062a00-8491-4099-9c55-1d1b64252939" containerID="63d4eb27736145f7904f2e40c8097cf8a628b27eaf1d6db8b8431222c674e54f" exitCode=0
Jul 28 22:34:24.715560 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.715383    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"
Jul 28 22:34:24.715560 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.715404    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9" event={"ID":"93062a00-8491-4099-9c55-1d1b64252939","Type":"ContainerDied","Data":"63d4eb27736145f7904f2e40c8097cf8a628b27eaf1d6db8b8431222c674e54f"}
Jul 28 22:34:24.715560 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.715456    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9" event={"ID":"93062a00-8491-4099-9c55-1d1b64252939","Type":"ContainerDied","Data":"bc4b47062f235072728cf87d9c34a86a5c5e56c6bb3658104886ebbd4d0e21a8"}
Jul 28 22:34:24.715560 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.715478    2574 scope.go:117] "RemoveContainer" containerID="63d4eb27736145f7904f2e40c8097cf8a628b27eaf1d6db8b8431222c674e54f"
Jul 28 22:34:24.725071 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.725056    2574 scope.go:117] "RemoveContainer" containerID="674c2786b10331adfecfaa58a03454a78ee7e5e5137bcd7bac0ceb6eb40e259d"
Jul 28 22:34:24.732855 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.732831    2574 scope.go:117] "RemoveContainer" containerID="63d4eb27736145f7904f2e40c8097cf8a628b27eaf1d6db8b8431222c674e54f"
Jul 28 22:34:24.733134 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:34:24.733111    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63d4eb27736145f7904f2e40c8097cf8a628b27eaf1d6db8b8431222c674e54f\": container with ID starting with 63d4eb27736145f7904f2e40c8097cf8a628b27eaf1d6db8b8431222c674e54f not found: ID does not exist" containerID="63d4eb27736145f7904f2e40c8097cf8a628b27eaf1d6db8b8431222c674e54f"
Jul 28 22:34:24.733207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.733167    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63d4eb27736145f7904f2e40c8097cf8a628b27eaf1d6db8b8431222c674e54f"} err="failed to get container status \"63d4eb27736145f7904f2e40c8097cf8a628b27eaf1d6db8b8431222c674e54f\": rpc error: code = NotFound desc = could not find container \"63d4eb27736145f7904f2e40c8097cf8a628b27eaf1d6db8b8431222c674e54f\": container with ID starting with 63d4eb27736145f7904f2e40c8097cf8a628b27eaf1d6db8b8431222c674e54f not found: ID does not exist"
Jul 28 22:34:24.733207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.733193    2574 scope.go:117] "RemoveContainer" containerID="674c2786b10331adfecfaa58a03454a78ee7e5e5137bcd7bac0ceb6eb40e259d"
Jul 28 22:34:24.733672 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:34:24.733532    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"674c2786b10331adfecfaa58a03454a78ee7e5e5137bcd7bac0ceb6eb40e259d\": container with ID starting with 674c2786b10331adfecfaa58a03454a78ee7e5e5137bcd7bac0ceb6eb40e259d not found: ID does not exist" containerID="674c2786b10331adfecfaa58a03454a78ee7e5e5137bcd7bac0ceb6eb40e259d"
Jul 28 22:34:24.733672 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.733595    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"674c2786b10331adfecfaa58a03454a78ee7e5e5137bcd7bac0ceb6eb40e259d"} err="failed to get container status \"674c2786b10331adfecfaa58a03454a78ee7e5e5137bcd7bac0ceb6eb40e259d\": rpc error: code = NotFound desc = could not find container \"674c2786b10331adfecfaa58a03454a78ee7e5e5137bcd7bac0ceb6eb40e259d\": container with ID starting with 674c2786b10331adfecfaa58a03454a78ee7e5e5137bcd7bac0ceb6eb40e259d not found: ID does not exist"
Jul 28 22:34:24.735959 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.735935    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"]
Jul 28 22:34:24.740539 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:24.740517    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-f8361843-openshift-default-76d5f74788-vdlz9"]
Jul 28 22:34:25.497473 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:34:25.495151    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93062a00-8491-4099-9c55-1d1b64252939" path="/var/lib/kubelet/pods/93062a00-8491-4099-9c55-1d1b64252939/volumes"
Jul 28 22:36:12.150335 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.150259    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"]
Jul 28 22:36:12.150784 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.150598    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93062a00-8491-4099-9c55-1d1b64252939" containerName="istio-proxy"
Jul 28 22:36:12.150784 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.150609    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="93062a00-8491-4099-9c55-1d1b64252939" containerName="istio-proxy"
Jul 28 22:36:12.150784 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.150624    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93062a00-8491-4099-9c55-1d1b64252939" containerName="istio-proxy"
Jul 28 22:36:12.150784 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.150629    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="93062a00-8491-4099-9c55-1d1b64252939" containerName="istio-proxy"
Jul 28 22:36:12.150784 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.150638    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3b468e0-6be5-4c0b-969e-802c9a766794" containerName="istio-proxy"
Jul 28 22:36:12.150784 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.150643    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3b468e0-6be5-4c0b-969e-802c9a766794" containerName="istio-proxy"
Jul 28 22:36:12.150784 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.150652    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58ac102e-4961-4250-a49f-bafec0dc3e3d" containerName="istio-proxy"
Jul 28 22:36:12.150784 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.150657    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="58ac102e-4961-4250-a49f-bafec0dc3e3d" containerName="istio-proxy"
Jul 28 22:36:12.150784 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.150666    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3b468e0-6be5-4c0b-969e-802c9a766794" containerName="istio-proxy"
Jul 28 22:36:12.150784 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.150672    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3b468e0-6be5-4c0b-969e-802c9a766794" containerName="istio-proxy"
Jul 28 22:36:12.150784 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.150725    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="93062a00-8491-4099-9c55-1d1b64252939" containerName="istio-proxy"
Jul 28 22:36:12.150784 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.150735    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="93062a00-8491-4099-9c55-1d1b64252939" containerName="istio-proxy"
Jul 28 22:36:12.150784 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.150742    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3b468e0-6be5-4c0b-969e-802c9a766794" containerName="istio-proxy"
Jul 28 22:36:12.152733 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.152713    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.155870 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.155843    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e64758c0-openshift-default-dockercfg-mj54n\""
Jul 28 22:36:12.182690 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.182657    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"]
Jul 28 22:36:12.249385 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.249347    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-workload-socket\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.249385 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.249384    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-envoy\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.249627 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.249401    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-data\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.249627 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.249419    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istiod-ca-cert\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.249627 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.249491    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnmgv\" (UniqueName: \"kubernetes.io/projected/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-kube-api-access-fnmgv\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.249627 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.249585    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-podinfo\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.249627 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.249613    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-token\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.249812 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.249635    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-workload-certs\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.249812 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.249652    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-credential-socket\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.350020 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.349983    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-credential-socket\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.350220 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.350028    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-workload-socket\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.350220 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.350047    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-envoy\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.350220 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.350065    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-data\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.350220 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.350127    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istiod-ca-cert\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.350220 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.350186    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnmgv\" (UniqueName: \"kubernetes.io/projected/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-kube-api-access-fnmgv\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.350574 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.350292    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-podinfo\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.350574 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.350328    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-token\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.350574 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.350357    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-workload-certs\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.350574 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.350544    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-credential-socket\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.350747 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.350580    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-data\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.350813 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.350773    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-workload-socket\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.350813 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.350796    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-workload-certs\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.351031 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.351010    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istiod-ca-cert\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.352497 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.352478    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-envoy\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.352979 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.352950    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-podinfo\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.357934 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.357905    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-token\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.358031 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.357935    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnmgv\" (UniqueName: \"kubernetes.io/projected/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-kube-api-access-fnmgv\") pod \"e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") " pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.463900 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.463864    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:12.612354 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.612329    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"]
Jul 28 22:36:12.614613 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:36:12.614586    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda332f42d_2dc6_4391_80fc_7bb3aa73eb8e.slice/crio-21b5e37b98b71d338bb1d7698921ee2477c7561e4fd44870e215010a6abd2ef2 WatchSource:0}: Error finding container 21b5e37b98b71d338bb1d7698921ee2477c7561e4fd44870e215010a6abd2ef2: Status 404 returned error can't find the container with id 21b5e37b98b71d338bb1d7698921ee2477c7561e4fd44870e215010a6abd2ef2
Jul 28 22:36:12.616814 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.616783    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:36:12.616931 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.616848    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:36:12.616931 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:12.616878    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:36:13.229204 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:13.229165    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j" event={"ID":"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e","Type":"ContainerStarted","Data":"1d63c310bdcdfcc4de4c8c1ca3a0644c9844df340a1b1efebd4657eef7355932"}
Jul 28 22:36:13.229658 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:13.229221    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j" event={"ID":"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e","Type":"ContainerStarted","Data":"21b5e37b98b71d338bb1d7698921ee2477c7561e4fd44870e215010a6abd2ef2"}
Jul 28 22:36:13.254136 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:13.254074    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j" podStartSLOduration=1.254057321 podStartE2EDuration="1.254057321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:36:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:36:13.252785542 +0000 UTC m=+1902.236928165" watchObservedRunningTime="2026-07-28 22:36:13.254057321 +0000 UTC m=+1902.238199940"
Jul 28 22:36:13.464550 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:13.464508    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:13.470840 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:13.470806    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:14.233950 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:14.233916    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:14.234890 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:14.234869    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:15.350951 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:15.350918    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46"]
Jul 28 22:36:15.351595 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:15.351578    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3b468e0-6be5-4c0b-969e-802c9a766794" containerName="istio-proxy"
Jul 28 22:36:15.354006 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:15.353985    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46"
Jul 28 22:36:15.359588 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:15.359547    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e64758c0-serving-cert\""
Jul 28 22:36:15.366036 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:15.365995    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46"]
Jul 28 22:36:15.371610 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:15.371579    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksq4x\" (UniqueName: \"kubernetes.io/projected/c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4-kube-api-access-ksq4x\") pod \"maas-api-e2e-mt-e64758c0-78b965467c-4qc46\" (UID: \"c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46"
Jul 28 22:36:15.371737 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:15.371687    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4-openshift-service-ca\") pod \"maas-api-e2e-mt-e64758c0-78b965467c-4qc46\" (UID: \"c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46"
Jul 28 22:36:15.371780 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:15.371747    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4-maas-api-tls\") pod \"maas-api-e2e-mt-e64758c0-78b965467c-4qc46\" (UID: \"c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46"
Jul 28 22:36:15.472341 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:15.472299    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4-openshift-service-ca\") pod \"maas-api-e2e-mt-e64758c0-78b965467c-4qc46\" (UID: \"c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46"
Jul 28 22:36:15.472341 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:15.472344    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4-maas-api-tls\") pod \"maas-api-e2e-mt-e64758c0-78b965467c-4qc46\" (UID: \"c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46"
Jul 28 22:36:15.472606 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:15.472418    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksq4x\" (UniqueName: \"kubernetes.io/projected/c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4-kube-api-access-ksq4x\") pod \"maas-api-e2e-mt-e64758c0-78b965467c-4qc46\" (UID: \"c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46"
Jul 28 22:36:15.472606 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:36:15.472519    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-serving-cert: secret "maas-api-e2e-mt-e64758c0-serving-cert" not found
Jul 28 22:36:15.472732 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:36:15.472619    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4-maas-api-tls podName:c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4 nodeName:}" failed. No retries permitted until 2026-07-28 22:36:15.972599399 +0000 UTC m=+1904.956742000 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4-maas-api-tls") pod "maas-api-e2e-mt-e64758c0-78b965467c-4qc46" (UID: "c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4") : secret "maas-api-e2e-mt-e64758c0-serving-cert" not found
Jul 28 22:36:15.472971 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:15.472948    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4-openshift-service-ca\") pod \"maas-api-e2e-mt-e64758c0-78b965467c-4qc46\" (UID: \"c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46"
Jul 28 22:36:15.489123 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:15.489087    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksq4x\" (UniqueName: \"kubernetes.io/projected/c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4-kube-api-access-ksq4x\") pod \"maas-api-e2e-mt-e64758c0-78b965467c-4qc46\" (UID: \"c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46"
Jul 28 22:36:15.977451 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:15.977408    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4-maas-api-tls\") pod \"maas-api-e2e-mt-e64758c0-78b965467c-4qc46\" (UID: \"c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46"
Jul 28 22:36:15.980206 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:15.980179    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4-maas-api-tls\") pod \"maas-api-e2e-mt-e64758c0-78b965467c-4qc46\" (UID: \"c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46"
Jul 28 22:36:16.267034 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:16.266930    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46"
Jul 28 22:36:16.448590 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:16.448563    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46"]
Jul 28 22:36:16.451635 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:36:16.451598    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc902598e_5b8b_4bf4_91e1_3ec0e78e9fc4.slice/crio-684ebf5de24711ce16f11bd5042510ed5b111385b42a0d3872803f39bdfa14a8 WatchSource:0}: Error finding container 684ebf5de24711ce16f11bd5042510ed5b111385b42a0d3872803f39bdfa14a8: Status 404 returned error can't find the container with id 684ebf5de24711ce16f11bd5042510ed5b111385b42a0d3872803f39bdfa14a8
Jul 28 22:36:17.251585 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:17.251546    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46" event={"ID":"c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4","Type":"ContainerStarted","Data":"b346ede34006a58807f4a83b6045b3af0bad91c6f8fb68e1d079b9fa8bc616aa"}
Jul 28 22:36:17.251781 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:17.251619    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46" event={"ID":"c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4","Type":"ContainerStarted","Data":"684ebf5de24711ce16f11bd5042510ed5b111385b42a0d3872803f39bdfa14a8"}
Jul 28 22:36:17.251781 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:17.251738    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46"
Jul 28 22:36:17.272021 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:17.271907    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46" podStartSLOduration=1.705001475 podStartE2EDuration="2.271888474s" totalImagesPullingTime="566.886999ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:36:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:36:17.271710377 +0000 UTC m=+1906.255852998" watchObservedRunningTime="2026-07-28 22:36:17.271888474 +0000 UTC m=+1906.256031094"
Jul 28 22:36:23.264138 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:23.264100    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46"
Jul 28 22:36:27.302510 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:27.302477    2574 generic.go:356] "Generic (PLEG): container finished" podID="a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" containerID="1d63c310bdcdfcc4de4c8c1ca3a0644c9844df340a1b1efebd4657eef7355932" exitCode=0
Jul 28 22:36:27.302902 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:27.302521    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j" event={"ID":"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e","Type":"ContainerDied","Data":"1d63c310bdcdfcc4de4c8c1ca3a0644c9844df340a1b1efebd4657eef7355932"}
Jul 28 22:36:27.302944 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:27.302905    2574 scope.go:117] "RemoveContainer" containerID="1d63c310bdcdfcc4de4c8c1ca3a0644c9844df340a1b1efebd4657eef7355932"
Jul 28 22:36:27.303899 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:27.303871    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:36:27.303956 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:27.303943    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:36:27.303996 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:27.303976    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:36:28.309544 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:28.309512    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j" event={"ID":"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e","Type":"ContainerStarted","Data":"b731c3a511474c0852174b187e0e2c846759f40438bd7cf005e79dc5adefcbd9"}
Jul 28 22:36:28.464688 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:28.464640    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:28.465878 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:28.465848    2574 patch_prober.go:28] interesting pod/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 28 22:36:28.466028 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:28.465906    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j" podUID="a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 28 22:36:29.234372 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:29.234313    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:29.464606 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:29.464558    2574 patch_prober.go:28] interesting pod/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 28 22:36:29.464996 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:29.464631    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j" podUID="a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 28 22:36:30.464467 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:30.464425    2574 patch_prober.go:28] interesting pod/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 28 22:36:30.464660 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:30.464486    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j" podUID="a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 28 22:36:31.464691 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:31.464645    2574 patch_prober.go:28] interesting pod/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 28 22:36:31.465171 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:31.464716    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j" podUID="a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 28 22:36:32.468639 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:32.468602    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:32.469757 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:32.469734    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:36:58.476767 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:58.476726    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46"]
Jul 28 22:36:58.477511 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:58.477072    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46" podUID="c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4" containerName="maas-api" containerID="cri-o://b346ede34006a58807f4a83b6045b3af0bad91c6f8fb68e1d079b9fa8bc616aa" gracePeriod=30
Jul 28 22:36:58.745636 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:58.745611    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46"
Jul 28 22:36:58.789641 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:58.789610    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4-maas-api-tls\") pod \"c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4\" (UID: \"c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4\") "
Jul 28 22:36:58.789834 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:58.789706    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4-openshift-service-ca\") pod \"c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4\" (UID: \"c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4\") "
Jul 28 22:36:58.789834 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:58.789768    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksq4x\" (UniqueName: \"kubernetes.io/projected/c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4-kube-api-access-ksq4x\") pod \"c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4\" (UID: \"c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4\") "
Jul 28 22:36:58.790058 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:58.790032    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4" (UID: "c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:36:58.791798 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:58.791762    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4" (UID: "c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:36:58.791936 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:58.791904    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4-kube-api-access-ksq4x" (OuterVolumeSpecName: "kube-api-access-ksq4x") pod "c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4" (UID: "c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4"). InnerVolumeSpecName "kube-api-access-ksq4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:36:58.890551 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:58.890515    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksq4x\" (UniqueName: \"kubernetes.io/projected/c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4-kube-api-access-ksq4x\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:36:58.890551 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:58.890543    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4-maas-api-tls\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:36:58.890551 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:58.890552    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4-openshift-service-ca\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:36:59.455063 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:59.455021    2574 generic.go:356] "Generic (PLEG): container finished" podID="c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4" containerID="b346ede34006a58807f4a83b6045b3af0bad91c6f8fb68e1d079b9fa8bc616aa" exitCode=0
Jul 28 22:36:59.455063 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:59.455067    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46"
Jul 28 22:36:59.455355 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:59.455098    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46" event={"ID":"c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4","Type":"ContainerDied","Data":"b346ede34006a58807f4a83b6045b3af0bad91c6f8fb68e1d079b9fa8bc616aa"}
Jul 28 22:36:59.455355 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:59.455148    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46" event={"ID":"c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4","Type":"ContainerDied","Data":"684ebf5de24711ce16f11bd5042510ed5b111385b42a0d3872803f39bdfa14a8"}
Jul 28 22:36:59.455355 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:59.455171    2574 scope.go:117] "RemoveContainer" containerID="b346ede34006a58807f4a83b6045b3af0bad91c6f8fb68e1d079b9fa8bc616aa"
Jul 28 22:36:59.464910 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:59.464889    2574 scope.go:117] "RemoveContainer" containerID="b346ede34006a58807f4a83b6045b3af0bad91c6f8fb68e1d079b9fa8bc616aa"
Jul 28 22:36:59.465178 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:36:59.465161    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b346ede34006a58807f4a83b6045b3af0bad91c6f8fb68e1d079b9fa8bc616aa\": container with ID starting with b346ede34006a58807f4a83b6045b3af0bad91c6f8fb68e1d079b9fa8bc616aa not found: ID does not exist" containerID="b346ede34006a58807f4a83b6045b3af0bad91c6f8fb68e1d079b9fa8bc616aa"
Jul 28 22:36:59.465225 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:59.465189    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b346ede34006a58807f4a83b6045b3af0bad91c6f8fb68e1d079b9fa8bc616aa"} err="failed to get container status \"b346ede34006a58807f4a83b6045b3af0bad91c6f8fb68e1d079b9fa8bc616aa\": rpc error: code = NotFound desc = could not find container \"b346ede34006a58807f4a83b6045b3af0bad91c6f8fb68e1d079b9fa8bc616aa\": container with ID starting with b346ede34006a58807f4a83b6045b3af0bad91c6f8fb68e1d079b9fa8bc616aa not found: ID does not exist"
Jul 28 22:36:59.477424 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:59.477397    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46"]
Jul 28 22:36:59.481098 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:59.481070    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e64758c0-78b965467c-4qc46"]
Jul 28 22:36:59.493924 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:36:59.493895    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4" path="/var/lib/kubelet/pods/c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4/volumes"
Jul 28 22:37:05.026245 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:05.026198    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"]
Jul 28 22:37:05.026656 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:05.026426    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j" podUID="a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" containerName="istio-proxy" containerID="cri-o://b731c3a511474c0852174b187e0e2c846759f40438bd7cf005e79dc5adefcbd9" gracePeriod=30
Jul 28 22:37:10.276145 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.276119    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:37:10.401583 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.401486    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnmgv\" (UniqueName: \"kubernetes.io/projected/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-kube-api-access-fnmgv\") pod \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") "
Jul 28 22:37:10.401583 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.401559    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-workload-socket\") pod \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") "
Jul 28 22:37:10.401583 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.401577    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-workload-certs\") pod \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") "
Jul 28 22:37:10.401879 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.401599    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-envoy\") pod \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") "
Jul 28 22:37:10.401879 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.401614    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-data\") pod \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") "
Jul 28 22:37:10.401879 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.401630    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-podinfo\") pod \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") "
Jul 28 22:37:10.401879 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.401678    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istiod-ca-cert\") pod \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") "
Jul 28 22:37:10.401879 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.401746    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-token\") pod \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") "
Jul 28 22:37:10.401879 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.401778    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-credential-socket\") pod \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\" (UID: \"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e\") "
Jul 28 22:37:10.402188 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.401968    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" (UID: "a332f42d-2dc6-4391-80fc-7bb3aa73eb8e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:37:10.402188 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.401977    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" (UID: "a332f42d-2dc6-4391-80fc-7bb3aa73eb8e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:37:10.402188 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.402143    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-workload-socket\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:37:10.402188 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.402162    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-workload-certs\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:37:10.402496 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.402211    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" (UID: "a332f42d-2dc6-4391-80fc-7bb3aa73eb8e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:37:10.402496 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.402277    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" (UID: "a332f42d-2dc6-4391-80fc-7bb3aa73eb8e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:37:10.402693 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.402661    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-data" (OuterVolumeSpecName: "istio-data") pod "a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" (UID: "a332f42d-2dc6-4391-80fc-7bb3aa73eb8e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:37:10.403965 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.403941    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-kube-api-access-fnmgv" (OuterVolumeSpecName: "kube-api-access-fnmgv") pod "a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" (UID: "a332f42d-2dc6-4391-80fc-7bb3aa73eb8e"). InnerVolumeSpecName "kube-api-access-fnmgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:37:10.404066 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.403976    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" (UID: "a332f42d-2dc6-4391-80fc-7bb3aa73eb8e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:37:10.404066 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.403985    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-token" (OuterVolumeSpecName: "istio-token") pod "a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" (UID: "a332f42d-2dc6-4391-80fc-7bb3aa73eb8e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:37:10.404141 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.404076    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" (UID: "a332f42d-2dc6-4391-80fc-7bb3aa73eb8e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 22:37:10.503006 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.502972    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istiod-ca-cert\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:37:10.503006 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.502997    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-token\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:37:10.503006 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.503007    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-credential-socket\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:37:10.503273 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.503017    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnmgv\" (UniqueName: \"kubernetes.io/projected/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-kube-api-access-fnmgv\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:37:10.503273 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.503027    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-envoy\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:37:10.503273 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.503035    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-data\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:37:10.503273 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.503042    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e-istio-podinfo\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:37:10.505360 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.505330    2574 generic.go:356] "Generic (PLEG): container finished" podID="a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" containerID="b731c3a511474c0852174b187e0e2c846759f40438bd7cf005e79dc5adefcbd9" exitCode=0
Jul 28 22:37:10.505479 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.505377    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j" event={"ID":"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e","Type":"ContainerDied","Data":"b731c3a511474c0852174b187e0e2c846759f40438bd7cf005e79dc5adefcbd9"}
Jul 28 22:37:10.505479 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.505419    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j" event={"ID":"a332f42d-2dc6-4391-80fc-7bb3aa73eb8e","Type":"ContainerDied","Data":"21b5e37b98b71d338bb1d7698921ee2477c7561e4fd44870e215010a6abd2ef2"}
Jul 28 22:37:10.505479 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.505439    2574 scope.go:117] "RemoveContainer" containerID="b731c3a511474c0852174b187e0e2c846759f40438bd7cf005e79dc5adefcbd9"
Jul 28 22:37:10.505479 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.505383    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"
Jul 28 22:37:10.514803 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.514784    2574 scope.go:117] "RemoveContainer" containerID="1d63c310bdcdfcc4de4c8c1ca3a0644c9844df340a1b1efebd4657eef7355932"
Jul 28 22:37:10.522484 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.522467    2574 scope.go:117] "RemoveContainer" containerID="b731c3a511474c0852174b187e0e2c846759f40438bd7cf005e79dc5adefcbd9"
Jul 28 22:37:10.522732 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:37:10.522712    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b731c3a511474c0852174b187e0e2c846759f40438bd7cf005e79dc5adefcbd9\": container with ID starting with b731c3a511474c0852174b187e0e2c846759f40438bd7cf005e79dc5adefcbd9 not found: ID does not exist" containerID="b731c3a511474c0852174b187e0e2c846759f40438bd7cf005e79dc5adefcbd9"
Jul 28 22:37:10.522807 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.522745    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b731c3a511474c0852174b187e0e2c846759f40438bd7cf005e79dc5adefcbd9"} err="failed to get container status \"b731c3a511474c0852174b187e0e2c846759f40438bd7cf005e79dc5adefcbd9\": rpc error: code = NotFound desc = could not find container \"b731c3a511474c0852174b187e0e2c846759f40438bd7cf005e79dc5adefcbd9\": container with ID starting with b731c3a511474c0852174b187e0e2c846759f40438bd7cf005e79dc5adefcbd9 not found: ID does not exist"
Jul 28 22:37:10.522807 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.522769    2574 scope.go:117] "RemoveContainer" containerID="1d63c310bdcdfcc4de4c8c1ca3a0644c9844df340a1b1efebd4657eef7355932"
Jul 28 22:37:10.523027 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:37:10.523010    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d63c310bdcdfcc4de4c8c1ca3a0644c9844df340a1b1efebd4657eef7355932\": container with ID starting with 1d63c310bdcdfcc4de4c8c1ca3a0644c9844df340a1b1efebd4657eef7355932 not found: ID does not exist" containerID="1d63c310bdcdfcc4de4c8c1ca3a0644c9844df340a1b1efebd4657eef7355932"
Jul 28 22:37:10.523063 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.523033    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d63c310bdcdfcc4de4c8c1ca3a0644c9844df340a1b1efebd4657eef7355932"} err="failed to get container status \"1d63c310bdcdfcc4de4c8c1ca3a0644c9844df340a1b1efebd4657eef7355932\": rpc error: code = NotFound desc = could not find container \"1d63c310bdcdfcc4de4c8c1ca3a0644c9844df340a1b1efebd4657eef7355932\": container with ID starting with 1d63c310bdcdfcc4de4c8c1ca3a0644c9844df340a1b1efebd4657eef7355932 not found: ID does not exist"
Jul 28 22:37:10.526759 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.526736    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"]
Jul 28 22:37:10.530289 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:10.530267    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e64758c0-openshift-default-7cbd97b5cb-4tv4j"]
Jul 28 22:37:11.494037 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:11.494004    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" path="/var/lib/kubelet/pods/a332f42d-2dc6-4391-80fc-7bb3aa73eb8e/volumes"
Jul 28 22:37:25.218934 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.218847    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"]
Jul 28 22:37:25.219326 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.219220    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4" containerName="maas-api"
Jul 28 22:37:25.219326 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.219251    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4" containerName="maas-api"
Jul 28 22:37:25.219326 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.219271    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" containerName="istio-proxy"
Jul 28 22:37:25.219326 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.219277    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" containerName="istio-proxy"
Jul 28 22:37:25.219467 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.219348    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" containerName="istio-proxy"
Jul 28 22:37:25.219467 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.219356    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" containerName="istio-proxy"
Jul 28 22:37:25.219467 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.219364    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c902598e-5b8b-4bf4-91e1-3ec0e78e9fc4" containerName="maas-api"
Jul 28 22:37:25.222810 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.222784    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.224932 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.224909    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-01e687c7-openshift-default-dockercfg-ngttn\""
Jul 28 22:37:25.235671 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.235492    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"]
Jul 28 22:37:25.339277 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.339222    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-podinfo\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.339471 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.339331    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-workload-socket\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.339471 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.339379    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2m5v\" (UniqueName: \"kubernetes.io/projected/4751c869-c8c4-447f-81e3-b803fc5a914e-kube-api-access-d2m5v\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.339471 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.339447    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4751c869-c8c4-447f-81e3-b803fc5a914e-istiod-ca-cert\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.339613 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.339493    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-credential-socket\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.339660 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.339635    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-workload-certs\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.339713 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.339697    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-envoy\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.339753 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.339729    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-data\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.339787 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.339757    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-token\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.440191 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.440146    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-podinfo\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.440191 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.440189    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-workload-socket\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.440473 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.440211    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2m5v\" (UniqueName: \"kubernetes.io/projected/4751c869-c8c4-447f-81e3-b803fc5a914e-kube-api-access-d2m5v\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.440473 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.440273    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4751c869-c8c4-447f-81e3-b803fc5a914e-istiod-ca-cert\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.440473 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.440303    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-credential-socket\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.440473 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.440354    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-workload-certs\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.440473 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.440381    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-envoy\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.440473 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.440397    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-data\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.440473 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.440436    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-token\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.441062 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.441032    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-credential-socket\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.441191 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.441063    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-workload-certs\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.441191 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.441165    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-data\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.441364 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.441248    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-workload-socket\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.441938 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.441889    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4751c869-c8c4-447f-81e3-b803fc5a914e-istiod-ca-cert\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.443449 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.443421    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-podinfo\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.443840 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.443823    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-envoy\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.448467 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.448434    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-token\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.448625 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.448604    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2m5v\" (UniqueName: \"kubernetes.io/projected/4751c869-c8c4-447f-81e3-b803fc5a914e-kube-api-access-d2m5v\") pod \"e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") " pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.533673 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.533579    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:25.674896 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.674866    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"]
Jul 28 22:37:25.677510 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:37:25.677482    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4751c869_c8c4_447f_81e3_b803fc5a914e.slice/crio-25ce0e4ba13d9ab738643d4de75dac7fc3b3383a27f1c650d478427b30df9fdf WatchSource:0}: Error finding container 25ce0e4ba13d9ab738643d4de75dac7fc3b3383a27f1c650d478427b30df9fdf: Status 404 returned error can't find the container with id 25ce0e4ba13d9ab738643d4de75dac7fc3b3383a27f1c650d478427b30df9fdf
Jul 28 22:37:25.679664 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.679635    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:37:25.679743 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.679700    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:37:25.679743 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:25.679727    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:37:26.588405 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:26.588363    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n" event={"ID":"4751c869-c8c4-447f-81e3-b803fc5a914e","Type":"ContainerStarted","Data":"5f8c3294a90ea94c2aea6ce48c2da06c5b79563404edf9390d7328668b11fcb7"}
Jul 28 22:37:26.588816 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:26.588419    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n" event={"ID":"4751c869-c8c4-447f-81e3-b803fc5a914e","Type":"ContainerStarted","Data":"25ce0e4ba13d9ab738643d4de75dac7fc3b3383a27f1c650d478427b30df9fdf"}
Jul 28 22:37:26.614573 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:26.614526    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n" podStartSLOduration=1.614508764 podStartE2EDuration="1.614508764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:37:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:37:26.613550962 +0000 UTC m=+1975.597693581" watchObservedRunningTime="2026-07-28 22:37:26.614508764 +0000 UTC m=+1975.598651383"
Jul 28 22:37:27.534613 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:27.534574    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:27.539947 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:27.539917    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:27.594097 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:27.594059    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:27.594968 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:27.594946    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:41.667324 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:41.667289    2574 generic.go:356] "Generic (PLEG): container finished" podID="4751c869-c8c4-447f-81e3-b803fc5a914e" containerID="5f8c3294a90ea94c2aea6ce48c2da06c5b79563404edf9390d7328668b11fcb7" exitCode=0
Jul 28 22:37:41.667858 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:41.667366    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n" event={"ID":"4751c869-c8c4-447f-81e3-b803fc5a914e","Type":"ContainerDied","Data":"5f8c3294a90ea94c2aea6ce48c2da06c5b79563404edf9390d7328668b11fcb7"}
Jul 28 22:37:41.667928 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:41.667880    2574 scope.go:117] "RemoveContainer" containerID="5f8c3294a90ea94c2aea6ce48c2da06c5b79563404edf9390d7328668b11fcb7"
Jul 28 22:37:41.669004 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:41.668978    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:37:41.669112 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:41.669047    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:37:41.669112 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:41.669078    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:37:42.594154 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:42.594116    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:42.673759 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:42.673723    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n" event={"ID":"4751c869-c8c4-447f-81e3-b803fc5a914e","Type":"ContainerStarted","Data":"43748f3deb6c1b3a02e93f12f9a4868ff719dcffe43be21ba3a79cf34a7aa46f"}
Jul 28 22:37:43.534130 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:43.534095    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:43.535152 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:43.535128    2574 patch_prober.go:28] interesting pod/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Jul 28 22:37:43.535292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:43.535174    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n" podUID="4751c869-c8c4-447f-81e3-b803fc5a914e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Jul 28 22:37:44.534889 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:44.534852    2574 patch_prober.go:28] interesting pod/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Jul 28 22:37:44.535332 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:44.534909    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n" podUID="4751c869-c8c4-447f-81e3-b803fc5a914e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Jul 28 22:37:45.534614 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:45.534571    2574 patch_prober.go:28] interesting pod/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Jul 28 22:37:45.534869 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:45.534641    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n" podUID="4751c869-c8c4-447f-81e3-b803fc5a914e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Jul 28 22:37:46.537899 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:46.537865    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:46.538305 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:46.538145    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:46.538933 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:46.538914    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:37:56.294767 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.294731    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"]
Jul 28 22:37:56.295179 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.295158    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" containerName="istio-proxy"
Jul 28 22:37:56.295245 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.295184    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a332f42d-2dc6-4391-80fc-7bb3aa73eb8e" containerName="istio-proxy"
Jul 28 22:37:56.297628 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.297606    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:37:56.300311 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.300281    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-01e687c7\"/\"default-dockercfg-pwnn2\""
Jul 28 22:37:56.300450 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.300313    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-01e687c7\"/\"kube-root-ca.crt\""
Jul 28 22:37:56.300450 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.300361    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-01e687c7\"/\"openshift-service-ca.crt\""
Jul 28 22:37:56.300793 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.300772    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-01e687c7\"/\"e2e-lifecycle-model-01e687c7-kserve-self-signed-certs\""
Jul 28 22:37:56.306257 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.306222    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"]
Jul 28 22:37:56.435403 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.435365    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-tls-certs\") pod \"e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") " pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:37:56.435622 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.435436    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-home\") pod \"e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") " pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:37:56.435622 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.435462    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s7f9\" (UniqueName: \"kubernetes.io/projected/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-kube-api-access-6s7f9\") pod \"e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") " pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:37:56.435622 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.435518    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-model-cache\") pod \"e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") " pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:37:56.435622 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.435545    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-tmp-dir\") pod \"e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") " pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:37:56.435622 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.435607    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-dshm\") pod \"e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") " pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:37:56.536414 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.536375    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-dshm\") pod \"e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") " pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:37:56.536639 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.536445    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-tls-certs\") pod \"e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") " pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:37:56.536639 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.536478    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-home\") pod \"e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") " pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:37:56.536639 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.536496    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s7f9\" (UniqueName: \"kubernetes.io/projected/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-kube-api-access-6s7f9\") pod \"e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") " pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:37:56.536639 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.536532    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-model-cache\") pod \"e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") " pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:37:56.536639 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.536565    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-tmp-dir\") pod \"e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") " pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:37:56.536931 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.536908    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-tmp-dir\") pod \"e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") " pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:37:56.537195 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.537172    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-model-cache\") pod \"e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") " pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:37:56.537360 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.537342    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-home\") pod \"e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") " pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:37:56.538892 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.538869    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-dshm\") pod \"e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") " pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:37:56.539352 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.539336    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-tls-certs\") pod \"e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") " pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:37:56.544509 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.544489    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s7f9\" (UniqueName: \"kubernetes.io/projected/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-kube-api-access-6s7f9\") pod \"e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") " pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:37:56.608527 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.608439    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:37:56.751269 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:56.751224    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"]
Jul 28 22:37:56.752904 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:37:56.752869    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3380fe3a_bffe_4d7d_a719_f5ec1dad0966.slice/crio-b34570283cca6821d3a4777a6f1acedbeb2bc23301c6a25ecfbce020eca35d00 WatchSource:0}: Error finding container b34570283cca6821d3a4777a6f1acedbeb2bc23301c6a25ecfbce020eca35d00: Status 404 returned error can't find the container with id b34570283cca6821d3a4777a6f1acedbeb2bc23301c6a25ecfbce020eca35d00
Jul 28 22:37:57.750049 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:57.750004    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf" event={"ID":"3380fe3a-bffe-4d7d-a719-f5ec1dad0966","Type":"ContainerStarted","Data":"f459367429247b56db6307511233131dab2fde95c8bd042524fefe5915f9f3bd"}
Jul 28 22:37:57.750049 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:57.750057    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf" event={"ID":"3380fe3a-bffe-4d7d-a719-f5ec1dad0966","Type":"ContainerStarted","Data":"b34570283cca6821d3a4777a6f1acedbeb2bc23301c6a25ecfbce020eca35d00"}
Jul 28 22:37:57.768306 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:37:57.768255    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf" podStartSLOduration=1.768221013 podStartE2EDuration="1.768221013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:37:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:37:57.766755895 +0000 UTC m=+2006.750898514" watchObservedRunningTime="2026-07-28 22:37:57.768221013 +0000 UTC m=+2006.752363633"
Jul 28 22:38:06.608947 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:38:06.608907    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:38:06.609506 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:38:06.608964    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:38:06.621336 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:38:06.621311    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:38:06.804306 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:38:06.804276    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:38:46.945696 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:38:46.945659    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"]
Jul 28 22:39:01.965461 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:01.965356    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf" podUID="3380fe3a-bffe-4d7d-a719-f5ec1dad0966" containerName="main" containerID="cri-o://f459367429247b56db6307511233131dab2fde95c8bd042524fefe5915f9f3bd" gracePeriod=45
Jul 28 22:39:02.224521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:02.224461    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:39:02.243848 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:02.243794    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-tls-certs\") pod \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") "
Jul 28 22:39:02.244036 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:02.243908    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-model-cache\") pod \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") "
Jul 28 22:39:02.244036 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:02.243958    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-home\") pod \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") "
Jul 28 22:39:02.244036 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:02.243979    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-tmp-dir\") pod \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") "
Jul 28 22:39:02.244036 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:02.244007    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6s7f9\" (UniqueName: \"kubernetes.io/projected/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-kube-api-access-6s7f9\") pod \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") "
Jul 28 22:39:02.244036 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:02.244036    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-dshm\") pod \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\" (UID: \"3380fe3a-bffe-4d7d-a719-f5ec1dad0966\") "
Jul 28 22:39:02.244426 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:02.244399    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-home" (OuterVolumeSpecName: "home") pod "3380fe3a-bffe-4d7d-a719-f5ec1dad0966" (UID: "3380fe3a-bffe-4d7d-a719-f5ec1dad0966"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:39:02.244709 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:02.244668    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3380fe3a-bffe-4d7d-a719-f5ec1dad0966" (UID: "3380fe3a-bffe-4d7d-a719-f5ec1dad0966"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:39:02.244845 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:02.244772    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-model-cache" (OuterVolumeSpecName: "model-cache") pod "3380fe3a-bffe-4d7d-a719-f5ec1dad0966" (UID: "3380fe3a-bffe-4d7d-a719-f5ec1dad0966"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:39:02.247040 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:02.247014    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-dshm" (OuterVolumeSpecName: "dshm") pod "3380fe3a-bffe-4d7d-a719-f5ec1dad0966" (UID: "3380fe3a-bffe-4d7d-a719-f5ec1dad0966"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:39:02.247170 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:02.247041    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3380fe3a-bffe-4d7d-a719-f5ec1dad0966" (UID: "3380fe3a-bffe-4d7d-a719-f5ec1dad0966"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:39:02.247170 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:02.247113    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-kube-api-access-6s7f9" (OuterVolumeSpecName: "kube-api-access-6s7f9") pod "3380fe3a-bffe-4d7d-a719-f5ec1dad0966" (UID: "3380fe3a-bffe-4d7d-a719-f5ec1dad0966"). InnerVolumeSpecName "kube-api-access-6s7f9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:39:02.345164 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:02.345129    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6s7f9\" (UniqueName: \"kubernetes.io/projected/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-kube-api-access-6s7f9\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:39:02.345164 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:02.345164    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-dshm\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:39:02.345401 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:02.345178    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-tls-certs\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:39:02.345401 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:02.345191    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-model-cache\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:39:02.345401 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:02.345202    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-home\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:39:02.345401 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:02.345213    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3380fe3a-bffe-4d7d-a719-f5ec1dad0966-tmp-dir\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:39:03.066010 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:03.065975    2574 generic.go:356] "Generic (PLEG): container finished" podID="3380fe3a-bffe-4d7d-a719-f5ec1dad0966" containerID="f459367429247b56db6307511233131dab2fde95c8bd042524fefe5915f9f3bd" exitCode=0
Jul 28 22:39:03.066526 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:03.066024    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"
Jul 28 22:39:03.066526 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:03.066058    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf" event={"ID":"3380fe3a-bffe-4d7d-a719-f5ec1dad0966","Type":"ContainerDied","Data":"f459367429247b56db6307511233131dab2fde95c8bd042524fefe5915f9f3bd"}
Jul 28 22:39:03.066526 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:03.066104    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf" event={"ID":"3380fe3a-bffe-4d7d-a719-f5ec1dad0966","Type":"ContainerDied","Data":"b34570283cca6821d3a4777a6f1acedbeb2bc23301c6a25ecfbce020eca35d00"}
Jul 28 22:39:03.066526 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:03.066122    2574 scope.go:117] "RemoveContainer" containerID="f459367429247b56db6307511233131dab2fde95c8bd042524fefe5915f9f3bd"
Jul 28 22:39:03.082035 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:03.082015    2574 scope.go:117] "RemoveContainer" containerID="f459367429247b56db6307511233131dab2fde95c8bd042524fefe5915f9f3bd"
Jul 28 22:39:03.082320 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:39:03.082303    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f459367429247b56db6307511233131dab2fde95c8bd042524fefe5915f9f3bd\": container with ID starting with f459367429247b56db6307511233131dab2fde95c8bd042524fefe5915f9f3bd not found: ID does not exist" containerID="f459367429247b56db6307511233131dab2fde95c8bd042524fefe5915f9f3bd"
Jul 28 22:39:03.082399 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:03.082328    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f459367429247b56db6307511233131dab2fde95c8bd042524fefe5915f9f3bd"} err="failed to get container status \"f459367429247b56db6307511233131dab2fde95c8bd042524fefe5915f9f3bd\": rpc error: code = NotFound desc = could not find container \"f459367429247b56db6307511233131dab2fde95c8bd042524fefe5915f9f3bd\": container with ID starting with f459367429247b56db6307511233131dab2fde95c8bd042524fefe5915f9f3bd not found: ID does not exist"
Jul 28 22:39:03.090647 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:03.090621    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"]
Jul 28 22:39:03.095088 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:03.095062    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-01e687c7/e2e-lifecycle-model-01e687c7-kserve-79dc8bccff-rkdlf"]
Jul 28 22:39:03.495172 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:03.495132    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3380fe3a-bffe-4d7d-a719-f5ec1dad0966" path="/var/lib/kubelet/pods/3380fe3a-bffe-4d7d-a719-f5ec1dad0966/volumes"
Jul 28 22:39:30.607347 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:30.607309    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"]
Jul 28 22:39:30.607821 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:30.607614    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n" podUID="4751c869-c8c4-447f-81e3-b803fc5a914e" containerName="istio-proxy" containerID="cri-o://43748f3deb6c1b3a02e93f12f9a4868ff719dcffe43be21ba3a79cf34a7aa46f" gracePeriod=30
Jul 28 22:39:31.540992 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:31.540955    2574 patch_prober.go:28] interesting pod/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 22:39:31.541173 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:31.541011    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n" podUID="4751c869-c8c4-447f-81e3-b803fc5a914e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 22:39:35.906304 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:35.906278    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:39:36.061583 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.061548    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-workload-socket\") pod \"4751c869-c8c4-447f-81e3-b803fc5a914e\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") "
Jul 28 22:39:36.061742 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.061594    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-token\") pod \"4751c869-c8c4-447f-81e3-b803fc5a914e\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") "
Jul 28 22:39:36.061742 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.061637    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-workload-certs\") pod \"4751c869-c8c4-447f-81e3-b803fc5a914e\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") "
Jul 28 22:39:36.061742 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.061665    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-data\") pod \"4751c869-c8c4-447f-81e3-b803fc5a914e\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") "
Jul 28 22:39:36.061742 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.061706    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-podinfo\") pod \"4751c869-c8c4-447f-81e3-b803fc5a914e\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") "
Jul 28 22:39:36.061962 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.061756    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2m5v\" (UniqueName: \"kubernetes.io/projected/4751c869-c8c4-447f-81e3-b803fc5a914e-kube-api-access-d2m5v\") pod \"4751c869-c8c4-447f-81e3-b803fc5a914e\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") "
Jul 28 22:39:36.061962 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.061791    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-credential-socket\") pod \"4751c869-c8c4-447f-81e3-b803fc5a914e\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") "
Jul 28 22:39:36.061962 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.061805    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4751c869-c8c4-447f-81e3-b803fc5a914e" (UID: "4751c869-c8c4-447f-81e3-b803fc5a914e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:39:36.061962 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.061817    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-envoy\") pod \"4751c869-c8c4-447f-81e3-b803fc5a914e\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") "
Jul 28 22:39:36.061962 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.061876    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4751c869-c8c4-447f-81e3-b803fc5a914e-istiod-ca-cert\") pod \"4751c869-c8c4-447f-81e3-b803fc5a914e\" (UID: \"4751c869-c8c4-447f-81e3-b803fc5a914e\") "
Jul 28 22:39:36.061962 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.061931    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4751c869-c8c4-447f-81e3-b803fc5a914e" (UID: "4751c869-c8c4-447f-81e3-b803fc5a914e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:39:36.062342 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.062092    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4751c869-c8c4-447f-81e3-b803fc5a914e" (UID: "4751c869-c8c4-447f-81e3-b803fc5a914e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:39:36.062342 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.062260    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-credential-socket\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:39:36.062342 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.062282    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-workload-socket\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:39:36.062342 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.062295    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-workload-certs\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:39:36.062549 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.062425    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4751c869-c8c4-447f-81e3-b803fc5a914e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4751c869-c8c4-447f-81e3-b803fc5a914e" (UID: "4751c869-c8c4-447f-81e3-b803fc5a914e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:39:36.062608 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.062578    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-data" (OuterVolumeSpecName: "istio-data") pod "4751c869-c8c4-447f-81e3-b803fc5a914e" (UID: "4751c869-c8c4-447f-81e3-b803fc5a914e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:39:36.064012 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.063987    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4751c869-c8c4-447f-81e3-b803fc5a914e-kube-api-access-d2m5v" (OuterVolumeSpecName: "kube-api-access-d2m5v") pod "4751c869-c8c4-447f-81e3-b803fc5a914e" (UID: "4751c869-c8c4-447f-81e3-b803fc5a914e"). InnerVolumeSpecName "kube-api-access-d2m5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:39:36.064111 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.063988    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4751c869-c8c4-447f-81e3-b803fc5a914e" (UID: "4751c869-c8c4-447f-81e3-b803fc5a914e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 22:39:36.064268 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.064247    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-token" (OuterVolumeSpecName: "istio-token") pod "4751c869-c8c4-447f-81e3-b803fc5a914e" (UID: "4751c869-c8c4-447f-81e3-b803fc5a914e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:39:36.064345 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.064253    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4751c869-c8c4-447f-81e3-b803fc5a914e" (UID: "4751c869-c8c4-447f-81e3-b803fc5a914e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:39:36.163141 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.163055    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-token\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:39:36.163141 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.163084    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-data\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:39:36.163141 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.163093    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-podinfo\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:39:36.163141 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.163102    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2m5v\" (UniqueName: \"kubernetes.io/projected/4751c869-c8c4-447f-81e3-b803fc5a914e-kube-api-access-d2m5v\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:39:36.163141 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.163113    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4751c869-c8c4-447f-81e3-b803fc5a914e-istio-envoy\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:39:36.163141 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.163120    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4751c869-c8c4-447f-81e3-b803fc5a914e-istiod-ca-cert\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:39:36.223776 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.223741    2574 generic.go:356] "Generic (PLEG): container finished" podID="4751c869-c8c4-447f-81e3-b803fc5a914e" containerID="43748f3deb6c1b3a02e93f12f9a4868ff719dcffe43be21ba3a79cf34a7aa46f" exitCode=0
Jul 28 22:39:36.223966 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.223792    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"
Jul 28 22:39:36.223966 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.223823    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n" event={"ID":"4751c869-c8c4-447f-81e3-b803fc5a914e","Type":"ContainerDied","Data":"43748f3deb6c1b3a02e93f12f9a4868ff719dcffe43be21ba3a79cf34a7aa46f"}
Jul 28 22:39:36.223966 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.223891    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n" event={"ID":"4751c869-c8c4-447f-81e3-b803fc5a914e","Type":"ContainerDied","Data":"25ce0e4ba13d9ab738643d4de75dac7fc3b3383a27f1c650d478427b30df9fdf"}
Jul 28 22:39:36.223966 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.223921    2574 scope.go:117] "RemoveContainer" containerID="43748f3deb6c1b3a02e93f12f9a4868ff719dcffe43be21ba3a79cf34a7aa46f"
Jul 28 22:39:36.232846 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.232828    2574 scope.go:117] "RemoveContainer" containerID="5f8c3294a90ea94c2aea6ce48c2da06c5b79563404edf9390d7328668b11fcb7"
Jul 28 22:39:36.240873 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.240855    2574 scope.go:117] "RemoveContainer" containerID="43748f3deb6c1b3a02e93f12f9a4868ff719dcffe43be21ba3a79cf34a7aa46f"
Jul 28 22:39:36.241147 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:39:36.241127    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43748f3deb6c1b3a02e93f12f9a4868ff719dcffe43be21ba3a79cf34a7aa46f\": container with ID starting with 43748f3deb6c1b3a02e93f12f9a4868ff719dcffe43be21ba3a79cf34a7aa46f not found: ID does not exist" containerID="43748f3deb6c1b3a02e93f12f9a4868ff719dcffe43be21ba3a79cf34a7aa46f"
Jul 28 22:39:36.241192 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.241157    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43748f3deb6c1b3a02e93f12f9a4868ff719dcffe43be21ba3a79cf34a7aa46f"} err="failed to get container status \"43748f3deb6c1b3a02e93f12f9a4868ff719dcffe43be21ba3a79cf34a7aa46f\": rpc error: code = NotFound desc = could not find container \"43748f3deb6c1b3a02e93f12f9a4868ff719dcffe43be21ba3a79cf34a7aa46f\": container with ID starting with 43748f3deb6c1b3a02e93f12f9a4868ff719dcffe43be21ba3a79cf34a7aa46f not found: ID does not exist"
Jul 28 22:39:36.241192 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.241175    2574 scope.go:117] "RemoveContainer" containerID="5f8c3294a90ea94c2aea6ce48c2da06c5b79563404edf9390d7328668b11fcb7"
Jul 28 22:39:36.241448 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:39:36.241430    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f8c3294a90ea94c2aea6ce48c2da06c5b79563404edf9390d7328668b11fcb7\": container with ID starting with 5f8c3294a90ea94c2aea6ce48c2da06c5b79563404edf9390d7328668b11fcb7 not found: ID does not exist" containerID="5f8c3294a90ea94c2aea6ce48c2da06c5b79563404edf9390d7328668b11fcb7"
Jul 28 22:39:36.241488 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.241454    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f8c3294a90ea94c2aea6ce48c2da06c5b79563404edf9390d7328668b11fcb7"} err="failed to get container status \"5f8c3294a90ea94c2aea6ce48c2da06c5b79563404edf9390d7328668b11fcb7\": rpc error: code = NotFound desc = could not find container \"5f8c3294a90ea94c2aea6ce48c2da06c5b79563404edf9390d7328668b11fcb7\": container with ID starting with 5f8c3294a90ea94c2aea6ce48c2da06c5b79563404edf9390d7328668b11fcb7 not found: ID does not exist"
Jul 28 22:39:36.248249 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.248211    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"]
Jul 28 22:39:36.255438 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:36.255407    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-01e687c7-openshift-default-876fb7844-wdr2n"]
Jul 28 22:39:37.493674 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:39:37.493644    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4751c869-c8c4-447f-81e3-b803fc5a914e" path="/var/lib/kubelet/pods/4751c869-c8c4-447f-81e3-b803fc5a914e/volumes"
Jul 28 22:40:26.636129 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.636041    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"]
Jul 28 22:40:26.636648 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.636581    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3380fe3a-bffe-4d7d-a719-f5ec1dad0966" containerName="main"
Jul 28 22:40:26.636648 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.636601    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3380fe3a-bffe-4d7d-a719-f5ec1dad0966" containerName="main"
Jul 28 22:40:26.636648 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.636613    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4751c869-c8c4-447f-81e3-b803fc5a914e" containerName="istio-proxy"
Jul 28 22:40:26.636648 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.636622    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4751c869-c8c4-447f-81e3-b803fc5a914e" containerName="istio-proxy"
Jul 28 22:40:26.636648 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.636639    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4751c869-c8c4-447f-81e3-b803fc5a914e" containerName="istio-proxy"
Jul 28 22:40:26.636648 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.636647    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4751c869-c8c4-447f-81e3-b803fc5a914e" containerName="istio-proxy"
Jul 28 22:40:26.636946 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.636752    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4751c869-c8c4-447f-81e3-b803fc5a914e" containerName="istio-proxy"
Jul 28 22:40:26.636946 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.636764    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4751c869-c8c4-447f-81e3-b803fc5a914e" containerName="istio-proxy"
Jul 28 22:40:26.636946 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.636776    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3380fe3a-bffe-4d7d-a719-f5ec1dad0966" containerName="main"
Jul 28 22:40:26.639729 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.639709    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.641726 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.641706    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-00957a-openshift-default-dockercfg-sqmb9\""
Jul 28 22:40:26.652871 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.652846    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"]
Jul 28 22:40:26.800709 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.800672    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-token\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.800709 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.800712    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-workload-certs\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.800957 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.800780    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-podinfo\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.800957 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.800818    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j9gj\" (UniqueName: \"kubernetes.io/projected/9ac63970-704b-4ec0-bd92-1b0230b2377c-kube-api-access-8j9gj\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.800957 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.800854    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-data\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.800957 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.800928    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-envoy\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.800957 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.800956    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ac63970-704b-4ec0-bd92-1b0230b2377c-istiod-ca-cert\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.801143 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.800982    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-workload-socket\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.801143 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.801050    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-credential-socket\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.902571 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.902489    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-credential-socket\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.902571 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.902531    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-token\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.902571 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.902558    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-workload-certs\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.902858 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.902603    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-podinfo\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.902858 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.902638    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j9gj\" (UniqueName: \"kubernetes.io/projected/9ac63970-704b-4ec0-bd92-1b0230b2377c-kube-api-access-8j9gj\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.902858 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.902673    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-data\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.902858 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.902727    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-envoy\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.902858 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.902753    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ac63970-704b-4ec0-bd92-1b0230b2377c-istiod-ca-cert\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.902858 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.902791    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-workload-socket\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.903165 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.903003    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-credential-socket\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.903165 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.903060    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-workload-certs\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.903165 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.903059    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-data\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.903165 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.903080    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-workload-socket\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.903544 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.903525    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ac63970-704b-4ec0-bd92-1b0230b2377c-istiod-ca-cert\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.904967 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.904943    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-envoy\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.905207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.905190    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-podinfo\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.909974 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.909955    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-token\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.910098 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.910020    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j9gj\" (UniqueName: \"kubernetes.io/projected/9ac63970-704b-4ec0-bd92-1b0230b2377c-kube-api-access-8j9gj\") pod \"e2e-inf-a-00957a-openshift-default-5466df956b-x287t\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") " pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:26.950004 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:26.949974    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:27.090660 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:27.090634    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"]
Jul 28 22:40:27.093044 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:40:27.093017    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ac63970_704b_4ec0_bd92_1b0230b2377c.slice/crio-4f3359ae7644122d742cce31b075d261dc5ca6fdecb30cdcb4fb9efc9c5cbbba WatchSource:0}: Error finding container 4f3359ae7644122d742cce31b075d261dc5ca6fdecb30cdcb4fb9efc9c5cbbba: Status 404 returned error can't find the container with id 4f3359ae7644122d742cce31b075d261dc5ca6fdecb30cdcb4fb9efc9c5cbbba
Jul 28 22:40:27.095070 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:27.095042    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:40:27.095181 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:27.095105    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:40:27.095181 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:27.095135    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:40:27.466871 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:27.466833    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t" event={"ID":"9ac63970-704b-4ec0-bd92-1b0230b2377c","Type":"ContainerStarted","Data":"df7fd4b85a929f5cf04db83d1e605c13301c195178955a0f59640f9937a1fe50"}
Jul 28 22:40:27.467053 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:27.466934    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t" event={"ID":"9ac63970-704b-4ec0-bd92-1b0230b2377c","Type":"ContainerStarted","Data":"4f3359ae7644122d742cce31b075d261dc5ca6fdecb30cdcb4fb9efc9c5cbbba"}
Jul 28 22:40:27.486025 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:27.485978    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t" podStartSLOduration=1.4859616629999999 podStartE2EDuration="1.485961663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:40:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:40:27.484616386 +0000 UTC m=+2156.468759015" watchObservedRunningTime="2026-07-28 22:40:27.485961663 +0000 UTC m=+2156.470104282"
Jul 28 22:40:27.950863 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:27.950832    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:28.955636 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:28.955599    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:29.476292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:29.476264    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:29.477350 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:29.477328    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:29.505584 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:29.505552    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j"]
Jul 28 22:40:29.509647 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:29.509619    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j"
Jul 28 22:40:29.511620 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:29.511594    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-00957a-serving-cert\""
Jul 28 22:40:29.517595 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:29.517568    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j"]
Jul 28 22:40:29.524270 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:29.524037    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a211f52a-eef1-41b2-9022-51d70a6f4876-maas-api-tls\") pod \"maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j\" (UID: \"a211f52a-eef1-41b2-9022-51d70a6f4876\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j"
Jul 28 22:40:29.524270 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:29.524096    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrsmf\" (UniqueName: \"kubernetes.io/projected/a211f52a-eef1-41b2-9022-51d70a6f4876-kube-api-access-jrsmf\") pod \"maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j\" (UID: \"a211f52a-eef1-41b2-9022-51d70a6f4876\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j"
Jul 28 22:40:29.524270 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:29.524164    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a211f52a-eef1-41b2-9022-51d70a6f4876-openshift-service-ca\") pod \"maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j\" (UID: \"a211f52a-eef1-41b2-9022-51d70a6f4876\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j"
Jul 28 22:40:29.625138 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:29.625099    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a211f52a-eef1-41b2-9022-51d70a6f4876-maas-api-tls\") pod \"maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j\" (UID: \"a211f52a-eef1-41b2-9022-51d70a6f4876\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j"
Jul 28 22:40:29.625370 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:29.625157    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrsmf\" (UniqueName: \"kubernetes.io/projected/a211f52a-eef1-41b2-9022-51d70a6f4876-kube-api-access-jrsmf\") pod \"maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j\" (UID: \"a211f52a-eef1-41b2-9022-51d70a6f4876\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j"
Jul 28 22:40:29.625370 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:29.625214    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a211f52a-eef1-41b2-9022-51d70a6f4876-openshift-service-ca\") pod \"maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j\" (UID: \"a211f52a-eef1-41b2-9022-51d70a6f4876\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j"
Jul 28 22:40:29.625370 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:40:29.625268    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-serving-cert: secret "maas-api-e2e-inf-a-00957a-serving-cert" not found
Jul 28 22:40:29.625370 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:40:29.625338    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a211f52a-eef1-41b2-9022-51d70a6f4876-maas-api-tls podName:a211f52a-eef1-41b2-9022-51d70a6f4876 nodeName:}" failed. No retries permitted until 2026-07-28 22:40:30.125317938 +0000 UTC m=+2159.109460535 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a211f52a-eef1-41b2-9022-51d70a6f4876-maas-api-tls") pod "maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j" (UID: "a211f52a-eef1-41b2-9022-51d70a6f4876") : secret "maas-api-e2e-inf-a-00957a-serving-cert" not found
Jul 28 22:40:29.626065 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:29.626044    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a211f52a-eef1-41b2-9022-51d70a6f4876-openshift-service-ca\") pod \"maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j\" (UID: \"a211f52a-eef1-41b2-9022-51d70a6f4876\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j"
Jul 28 22:40:29.636216 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:29.636184    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrsmf\" (UniqueName: \"kubernetes.io/projected/a211f52a-eef1-41b2-9022-51d70a6f4876-kube-api-access-jrsmf\") pod \"maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j\" (UID: \"a211f52a-eef1-41b2-9022-51d70a6f4876\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j"
Jul 28 22:40:30.128694 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:30.128652    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a211f52a-eef1-41b2-9022-51d70a6f4876-maas-api-tls\") pod \"maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j\" (UID: \"a211f52a-eef1-41b2-9022-51d70a6f4876\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j"
Jul 28 22:40:30.130942 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:30.130919    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a211f52a-eef1-41b2-9022-51d70a6f4876-maas-api-tls\") pod \"maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j\" (UID: \"a211f52a-eef1-41b2-9022-51d70a6f4876\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j"
Jul 28 22:40:30.426298 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:30.426206    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j"
Jul 28 22:40:30.782019 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:30.781992    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j"]
Jul 28 22:40:30.783738 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:40:30.783712    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda211f52a_eef1_41b2_9022_51d70a6f4876.slice/crio-e502e5b69c4b298fb50cf60ec8ee84c6df2a6f4a5b1f7ec76f2fc6c200591768 WatchSource:0}: Error finding container e502e5b69c4b298fb50cf60ec8ee84c6df2a6f4a5b1f7ec76f2fc6c200591768: Status 404 returned error can't find the container with id e502e5b69c4b298fb50cf60ec8ee84c6df2a6f4a5b1f7ec76f2fc6c200591768
Jul 28 22:40:30.785068 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:30.785043    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 22:40:31.500081 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:31.500053    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j"
Jul 28 22:40:31.500462 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:31.500098    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j" event={"ID":"a211f52a-eef1-41b2-9022-51d70a6f4876","Type":"ContainerStarted","Data":"796a0605ee8790ebd787a314aefe77251fe7aecdace00e3ca06f22d582534aa2"}
Jul 28 22:40:31.500462 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:31.500119    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j" event={"ID":"a211f52a-eef1-41b2-9022-51d70a6f4876","Type":"ContainerStarted","Data":"e502e5b69c4b298fb50cf60ec8ee84c6df2a6f4a5b1f7ec76f2fc6c200591768"}
Jul 28 22:40:31.513982 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:31.513942    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j" podStartSLOduration=1.979778804 podStartE2EDuration="2.513928634s" totalImagesPullingTime="534.14983ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:40:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:40:31.513821814 +0000 UTC m=+2160.497964432" watchObservedRunningTime="2026-07-28 22:40:31.513928634 +0000 UTC m=+2160.498071254"
Jul 28 22:40:37.509701 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:37.509672    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j"
Jul 28 22:40:44.477335 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:44.477297    2574 patch_prober.go:28] interesting pod/e2e-inf-a-00957a-openshift-default-5466df956b-x287t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 28 22:40:44.477776 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:44.477365    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t" podUID="9ac63970-704b-4ec0-bd92-1b0230b2377c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 28 22:40:44.557940 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:44.557906    2574 generic.go:356] "Generic (PLEG): container finished" podID="9ac63970-704b-4ec0-bd92-1b0230b2377c" containerID="df7fd4b85a929f5cf04db83d1e605c13301c195178955a0f59640f9937a1fe50" exitCode=0
Jul 28 22:40:44.558135 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:44.557954    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t" event={"ID":"9ac63970-704b-4ec0-bd92-1b0230b2377c","Type":"ContainerDied","Data":"df7fd4b85a929f5cf04db83d1e605c13301c195178955a0f59640f9937a1fe50"}
Jul 28 22:40:44.558410 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:44.558380    2574 scope.go:117] "RemoveContainer" containerID="df7fd4b85a929f5cf04db83d1e605c13301c195178955a0f59640f9937a1fe50"
Jul 28 22:40:44.559417 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:44.559386    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:40:44.559496 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:44.559459    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:40:44.559539 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:44.559496    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:40:45.567165 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:45.567134    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t" event={"ID":"9ac63970-704b-4ec0-bd92-1b0230b2377c","Type":"ContainerStarted","Data":"b1ab26ede5b649bd06ec88d5e890476bf46be2b80b89978201e07e54e2a2995a"}
Jul 28 22:40:45.951301 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:45.950842    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:45.951301 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:45.950975    2574 patch_prober.go:28] interesting pod/e2e-inf-a-00957a-openshift-default-5466df956b-x287t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 28 22:40:45.951301 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:45.951016    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t" podUID="9ac63970-704b-4ec0-bd92-1b0230b2377c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 28 22:40:46.951329 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:46.951285    2574 patch_prober.go:28] interesting pod/e2e-inf-a-00957a-openshift-default-5466df956b-x287t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 28 22:40:46.951838 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:46.951353    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t" podUID="9ac63970-704b-4ec0-bd92-1b0230b2377c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 28 22:40:47.956412 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:47.956363    2574 patch_prober.go:28] interesting pod/e2e-inf-a-00957a-openshift-default-5466df956b-x287t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 28 22:40:47.956905 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:47.956435    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t" podUID="9ac63970-704b-4ec0-bd92-1b0230b2377c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 28 22:40:48.951276 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:48.950773    2574 patch_prober.go:28] interesting pod/e2e-inf-a-00957a-openshift-default-5466df956b-x287t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 28 22:40:48.951276 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:48.950837    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t" podUID="9ac63970-704b-4ec0-bd92-1b0230b2377c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 28 22:40:49.954794 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:49.954762    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:49.955210 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:49.955056    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:40:49.955614 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:40:49.955595    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:41:00.878531 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:00.878492    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l"]
Jul 28 22:41:00.881668 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:00.881646    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l"
Jul 28 22:41:00.883607 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:00.883585    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-8a6fb9-serving-cert\""
Jul 28 22:41:00.891170 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:00.891147    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l"]
Jul 28 22:41:01.013056 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:01.013015    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xknj\" (UniqueName: \"kubernetes.io/projected/6a7c275d-20b7-4e4b-8216-a1970eb8fcd5-kube-api-access-6xknj\") pod \"maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l\" (UID: \"6a7c275d-20b7-4e4b-8216-a1970eb8fcd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l"
Jul 28 22:41:01.013226 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:01.013065    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a7c275d-20b7-4e4b-8216-a1970eb8fcd5-maas-api-tls\") pod \"maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l\" (UID: \"6a7c275d-20b7-4e4b-8216-a1970eb8fcd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l"
Jul 28 22:41:01.013226 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:01.013196    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a7c275d-20b7-4e4b-8216-a1970eb8fcd5-openshift-service-ca\") pod \"maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l\" (UID: \"6a7c275d-20b7-4e4b-8216-a1970eb8fcd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l"
Jul 28 22:41:01.114347 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:01.114312    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xknj\" (UniqueName: \"kubernetes.io/projected/6a7c275d-20b7-4e4b-8216-a1970eb8fcd5-kube-api-access-6xknj\") pod \"maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l\" (UID: \"6a7c275d-20b7-4e4b-8216-a1970eb8fcd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l"
Jul 28 22:41:01.114347 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:01.114353    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a7c275d-20b7-4e4b-8216-a1970eb8fcd5-maas-api-tls\") pod \"maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l\" (UID: \"6a7c275d-20b7-4e4b-8216-a1970eb8fcd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l"
Jul 28 22:41:01.114576 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:01.114405    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a7c275d-20b7-4e4b-8216-a1970eb8fcd5-openshift-service-ca\") pod \"maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l\" (UID: \"6a7c275d-20b7-4e4b-8216-a1970eb8fcd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l"
Jul 28 22:41:01.115024 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:01.115005    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a7c275d-20b7-4e4b-8216-a1970eb8fcd5-openshift-service-ca\") pod \"maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l\" (UID: \"6a7c275d-20b7-4e4b-8216-a1970eb8fcd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l"
Jul 28 22:41:01.116961 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:01.116937    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a7c275d-20b7-4e4b-8216-a1970eb8fcd5-maas-api-tls\") pod \"maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l\" (UID: \"6a7c275d-20b7-4e4b-8216-a1970eb8fcd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l"
Jul 28 22:41:01.121985 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:01.121964    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xknj\" (UniqueName: \"kubernetes.io/projected/6a7c275d-20b7-4e4b-8216-a1970eb8fcd5-kube-api-access-6xknj\") pod \"maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l\" (UID: \"6a7c275d-20b7-4e4b-8216-a1970eb8fcd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l"
Jul 28 22:41:01.192450 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:01.192421    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l"
Jul 28 22:41:01.326816 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:01.326790    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l"]
Jul 28 22:41:01.329317 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:41:01.329276    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a7c275d_20b7_4e4b_8216_a1970eb8fcd5.slice/crio-52ae95641415ca988c1df46c7f5563f62d46f72f58fb0f1faf4b732d50db1e71 WatchSource:0}: Error finding container 52ae95641415ca988c1df46c7f5563f62d46f72f58fb0f1faf4b732d50db1e71: Status 404 returned error can't find the container with id 52ae95641415ca988c1df46c7f5563f62d46f72f58fb0f1faf4b732d50db1e71
Jul 28 22:41:01.645544 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:01.645514    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l" event={"ID":"6a7c275d-20b7-4e4b-8216-a1970eb8fcd5","Type":"ContainerStarted","Data":"52ae95641415ca988c1df46c7f5563f62d46f72f58fb0f1faf4b732d50db1e71"}
Jul 28 22:41:02.653591 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:02.653555    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l" event={"ID":"6a7c275d-20b7-4e4b-8216-a1970eb8fcd5","Type":"ContainerStarted","Data":"765a8c1bea88e8cc0db1ec0c3e7dbe2558a6ac198712c8d1dfcaf64419bcf0ad"}
Jul 28 22:41:02.653993 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:02.653637    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l"
Jul 28 22:41:02.669603 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:02.669556    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l" podStartSLOduration=2.410328489 podStartE2EDuration="2.669539862s" totalImagesPullingTime="259.211373ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:41:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:41:02.668259609 +0000 UTC m=+2191.652402225" watchObservedRunningTime="2026-07-28 22:41:02.669539862 +0000 UTC m=+2191.653682480"
Jul 28 22:41:08.664268 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:08.664216    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l"
Jul 28 22:41:30.094009 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.093975    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"]
Jul 28 22:41:30.096471 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.096453    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:30.099046 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.099015    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-00957a\"/\"openshift-service-ca.crt\""
Jul 28 22:41:30.099198 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.099019    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-00957a\"/\"kube-root-ca.crt\""
Jul 28 22:41:30.099198 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.099020    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-00957a\"/\"test-model-00957a-kserve-self-signed-certs\""
Jul 28 22:41:30.099198 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.099020    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-00957a\"/\"default-dockercfg-tgcmr\""
Jul 28 22:41:30.107509 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.107486    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"]
Jul 28 22:41:30.182631 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.182595    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-model-cache\") pod \"test-model-00957a-kserve-7bf94b6f69-7f9gs\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") " pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:30.182840 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.182650    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-dshm\") pod \"test-model-00957a-kserve-7bf94b6f69-7f9gs\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") " pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:30.182840 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.182713    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-home\") pod \"test-model-00957a-kserve-7bf94b6f69-7f9gs\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") " pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:30.182840 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.182747    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-tmp-dir\") pod \"test-model-00957a-kserve-7bf94b6f69-7f9gs\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") " pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:30.182840 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.182792    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq98d\" (UniqueName: \"kubernetes.io/projected/8dd52709-2542-4c63-8aba-15dd2386f80d-kube-api-access-sq98d\") pod \"test-model-00957a-kserve-7bf94b6f69-7f9gs\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") " pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:30.182840 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.182818    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dd52709-2542-4c63-8aba-15dd2386f80d-tls-certs\") pod \"test-model-00957a-kserve-7bf94b6f69-7f9gs\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") " pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:30.283398 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.283359    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq98d\" (UniqueName: \"kubernetes.io/projected/8dd52709-2542-4c63-8aba-15dd2386f80d-kube-api-access-sq98d\") pod \"test-model-00957a-kserve-7bf94b6f69-7f9gs\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") " pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:30.283603 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.283405    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dd52709-2542-4c63-8aba-15dd2386f80d-tls-certs\") pod \"test-model-00957a-kserve-7bf94b6f69-7f9gs\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") " pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:30.283681 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.283592    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-model-cache\") pod \"test-model-00957a-kserve-7bf94b6f69-7f9gs\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") " pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:30.283681 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.283665    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-dshm\") pod \"test-model-00957a-kserve-7bf94b6f69-7f9gs\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") " pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:30.283780 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.283713    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-home\") pod \"test-model-00957a-kserve-7bf94b6f69-7f9gs\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") " pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:30.283780 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.283739    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-tmp-dir\") pod \"test-model-00957a-kserve-7bf94b6f69-7f9gs\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") " pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:30.283993 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.283963    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-model-cache\") pod \"test-model-00957a-kserve-7bf94b6f69-7f9gs\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") " pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:30.284112 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.283994    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-home\") pod \"test-model-00957a-kserve-7bf94b6f69-7f9gs\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") " pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:30.284158 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.284137    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-tmp-dir\") pod \"test-model-00957a-kserve-7bf94b6f69-7f9gs\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") " pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:30.285733 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.285714    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-dshm\") pod \"test-model-00957a-kserve-7bf94b6f69-7f9gs\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") " pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:30.286136 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.286116    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dd52709-2542-4c63-8aba-15dd2386f80d-tls-certs\") pod \"test-model-00957a-kserve-7bf94b6f69-7f9gs\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") " pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:30.290525 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.290505    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq98d\" (UniqueName: \"kubernetes.io/projected/8dd52709-2542-4c63-8aba-15dd2386f80d-kube-api-access-sq98d\") pod \"test-model-00957a-kserve-7bf94b6f69-7f9gs\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") " pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:30.409075 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.408987    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:30.537865 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.537831    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"]
Jul 28 22:41:30.539539 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:41:30.539505    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8dd52709_2542_4c63_8aba_15dd2386f80d.slice/crio-090e2b2fa5bb9deca0155f2bef9611d7b99052e7d69ea23b11100cd7dfb64736 WatchSource:0}: Error finding container 090e2b2fa5bb9deca0155f2bef9611d7b99052e7d69ea23b11100cd7dfb64736: Status 404 returned error can't find the container with id 090e2b2fa5bb9deca0155f2bef9611d7b99052e7d69ea23b11100cd7dfb64736
Jul 28 22:41:30.792250 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.792195    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs" event={"ID":"8dd52709-2542-4c63-8aba-15dd2386f80d","Type":"ContainerStarted","Data":"4f7ace9c2dcc88713be3a2456fca163e5b97fdc705d2cc15187dd08ff9752530"}
Jul 28 22:41:30.792430 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.792262    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs" event={"ID":"8dd52709-2542-4c63-8aba-15dd2386f80d","Type":"ContainerStarted","Data":"090e2b2fa5bb9deca0155f2bef9611d7b99052e7d69ea23b11100cd7dfb64736"}
Jul 28 22:41:30.808851 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:30.808806    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs" podStartSLOduration=0.808793475 podStartE2EDuration="808.793475ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:41:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:41:30.807577269 +0000 UTC m=+2219.791719886" watchObservedRunningTime="2026-07-28 22:41:30.808793475 +0000 UTC m=+2219.792936093"
Jul 28 22:41:40.410136 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:40.410074    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:40.410662 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:40.410157    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:40.422348 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:40.422327    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:41:40.850916 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:41:40.850885    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:42:50.588859 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:42:50.588808    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"]
Jul 28 22:43:02.200199 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:02.200163    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j"]
Jul 28 22:43:02.200820 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:02.200521    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j" podUID="a211f52a-eef1-41b2-9022-51d70a6f4876" containerName="maas-api" containerID="cri-o://796a0605ee8790ebd787a314aefe77251fe7aecdace00e3ca06f22d582534aa2" gracePeriod=30
Jul 28 22:43:02.504106 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:02.504071    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j" podUID="a211f52a-eef1-41b2-9022-51d70a6f4876" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.56:8443/health\": dial tcp 10.133.0.56:8443: connect: connection refused"
Jul 28 22:43:02.663503 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:02.663478    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j"
Jul 28 22:43:02.695566 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:02.695540    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a211f52a-eef1-41b2-9022-51d70a6f4876-openshift-service-ca\") pod \"a211f52a-eef1-41b2-9022-51d70a6f4876\" (UID: \"a211f52a-eef1-41b2-9022-51d70a6f4876\") "
Jul 28 22:43:02.695750 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:02.695595    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrsmf\" (UniqueName: \"kubernetes.io/projected/a211f52a-eef1-41b2-9022-51d70a6f4876-kube-api-access-jrsmf\") pod \"a211f52a-eef1-41b2-9022-51d70a6f4876\" (UID: \"a211f52a-eef1-41b2-9022-51d70a6f4876\") "
Jul 28 22:43:02.695750 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:02.695627    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a211f52a-eef1-41b2-9022-51d70a6f4876-maas-api-tls\") pod \"a211f52a-eef1-41b2-9022-51d70a6f4876\" (UID: \"a211f52a-eef1-41b2-9022-51d70a6f4876\") "
Jul 28 22:43:02.695949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:02.695918    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a211f52a-eef1-41b2-9022-51d70a6f4876-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a211f52a-eef1-41b2-9022-51d70a6f4876" (UID: "a211f52a-eef1-41b2-9022-51d70a6f4876"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:43:02.697805 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:02.697769    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a211f52a-eef1-41b2-9022-51d70a6f4876-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a211f52a-eef1-41b2-9022-51d70a6f4876" (UID: "a211f52a-eef1-41b2-9022-51d70a6f4876"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:43:02.697930 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:02.697829    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a211f52a-eef1-41b2-9022-51d70a6f4876-kube-api-access-jrsmf" (OuterVolumeSpecName: "kube-api-access-jrsmf") pod "a211f52a-eef1-41b2-9022-51d70a6f4876" (UID: "a211f52a-eef1-41b2-9022-51d70a6f4876"). InnerVolumeSpecName "kube-api-access-jrsmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:43:02.797289 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:02.797171    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a211f52a-eef1-41b2-9022-51d70a6f4876-openshift-service-ca\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:43:02.797289 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:02.797198    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrsmf\" (UniqueName: \"kubernetes.io/projected/a211f52a-eef1-41b2-9022-51d70a6f4876-kube-api-access-jrsmf\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:43:02.797289 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:02.797209    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a211f52a-eef1-41b2-9022-51d70a6f4876-maas-api-tls\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:43:03.231944 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:03.231912    2574 generic.go:356] "Generic (PLEG): container finished" podID="a211f52a-eef1-41b2-9022-51d70a6f4876" containerID="796a0605ee8790ebd787a314aefe77251fe7aecdace00e3ca06f22d582534aa2" exitCode=0
Jul 28 22:43:03.232430 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:03.231962    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j"
Jul 28 22:43:03.232430 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:03.231996    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j" event={"ID":"a211f52a-eef1-41b2-9022-51d70a6f4876","Type":"ContainerDied","Data":"796a0605ee8790ebd787a314aefe77251fe7aecdace00e3ca06f22d582534aa2"}
Jul 28 22:43:03.232430 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:03.232049    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j" event={"ID":"a211f52a-eef1-41b2-9022-51d70a6f4876","Type":"ContainerDied","Data":"e502e5b69c4b298fb50cf60ec8ee84c6df2a6f4a5b1f7ec76f2fc6c200591768"}
Jul 28 22:43:03.232430 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:03.232072    2574 scope.go:117] "RemoveContainer" containerID="796a0605ee8790ebd787a314aefe77251fe7aecdace00e3ca06f22d582534aa2"
Jul 28 22:43:03.241699 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:03.241613    2574 scope.go:117] "RemoveContainer" containerID="796a0605ee8790ebd787a314aefe77251fe7aecdace00e3ca06f22d582534aa2"
Jul 28 22:43:03.241934 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:43:03.241904    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"796a0605ee8790ebd787a314aefe77251fe7aecdace00e3ca06f22d582534aa2\": container with ID starting with 796a0605ee8790ebd787a314aefe77251fe7aecdace00e3ca06f22d582534aa2 not found: ID does not exist" containerID="796a0605ee8790ebd787a314aefe77251fe7aecdace00e3ca06f22d582534aa2"
Jul 28 22:43:03.242060 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:03.241944    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"796a0605ee8790ebd787a314aefe77251fe7aecdace00e3ca06f22d582534aa2"} err="failed to get container status \"796a0605ee8790ebd787a314aefe77251fe7aecdace00e3ca06f22d582534aa2\": rpc error: code = NotFound desc = could not find container \"796a0605ee8790ebd787a314aefe77251fe7aecdace00e3ca06f22d582534aa2\": container with ID starting with 796a0605ee8790ebd787a314aefe77251fe7aecdace00e3ca06f22d582534aa2 not found: ID does not exist"
Jul 28 22:43:03.256008 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:03.255985    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j"]
Jul 28 22:43:03.264586 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:03.264561    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-00957a-5879b5d7c7-tt79j"]
Jul 28 22:43:03.493275 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:03.493182    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a211f52a-eef1-41b2-9022-51d70a6f4876" path="/var/lib/kubelet/pods/a211f52a-eef1-41b2-9022-51d70a6f4876/volumes"
Jul 28 22:43:05.607667 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:05.607626    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs" podUID="8dd52709-2542-4c63-8aba-15dd2386f80d" containerName="main" containerID="cri-o://4f7ace9c2dcc88713be3a2456fca163e5b97fdc705d2cc15187dd08ff9752530" gracePeriod=45
Jul 28 22:43:05.857518 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:05.857494    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:43:05.926311 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:05.926216    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-dshm\") pod \"8dd52709-2542-4c63-8aba-15dd2386f80d\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") "
Jul 28 22:43:05.926311 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:05.926272    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dd52709-2542-4c63-8aba-15dd2386f80d-tls-certs\") pod \"8dd52709-2542-4c63-8aba-15dd2386f80d\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") "
Jul 28 22:43:05.926534 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:05.926320    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sq98d\" (UniqueName: \"kubernetes.io/projected/8dd52709-2542-4c63-8aba-15dd2386f80d-kube-api-access-sq98d\") pod \"8dd52709-2542-4c63-8aba-15dd2386f80d\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") "
Jul 28 22:43:05.926534 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:05.926359    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-model-cache\") pod \"8dd52709-2542-4c63-8aba-15dd2386f80d\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") "
Jul 28 22:43:05.926534 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:05.926406    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-home\") pod \"8dd52709-2542-4c63-8aba-15dd2386f80d\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") "
Jul 28 22:43:05.926534 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:05.926434    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-tmp-dir\") pod \"8dd52709-2542-4c63-8aba-15dd2386f80d\" (UID: \"8dd52709-2542-4c63-8aba-15dd2386f80d\") "
Jul 28 22:43:05.926752 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:05.926667    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-home" (OuterVolumeSpecName: "home") pod "8dd52709-2542-4c63-8aba-15dd2386f80d" (UID: "8dd52709-2542-4c63-8aba-15dd2386f80d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:43:05.926752 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:05.926683    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-model-cache" (OuterVolumeSpecName: "model-cache") pod "8dd52709-2542-4c63-8aba-15dd2386f80d" (UID: "8dd52709-2542-4c63-8aba-15dd2386f80d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:43:05.926859 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:05.926755    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8dd52709-2542-4c63-8aba-15dd2386f80d" (UID: "8dd52709-2542-4c63-8aba-15dd2386f80d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:43:05.928578 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:05.928526    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dd52709-2542-4c63-8aba-15dd2386f80d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8dd52709-2542-4c63-8aba-15dd2386f80d" (UID: "8dd52709-2542-4c63-8aba-15dd2386f80d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:43:05.928725 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:05.928599    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-dshm" (OuterVolumeSpecName: "dshm") pod "8dd52709-2542-4c63-8aba-15dd2386f80d" (UID: "8dd52709-2542-4c63-8aba-15dd2386f80d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:43:05.928725 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:05.928627    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dd52709-2542-4c63-8aba-15dd2386f80d-kube-api-access-sq98d" (OuterVolumeSpecName: "kube-api-access-sq98d") pod "8dd52709-2542-4c63-8aba-15dd2386f80d" (UID: "8dd52709-2542-4c63-8aba-15dd2386f80d"). InnerVolumeSpecName "kube-api-access-sq98d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:43:06.027682 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:06.027649    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq98d\" (UniqueName: \"kubernetes.io/projected/8dd52709-2542-4c63-8aba-15dd2386f80d-kube-api-access-sq98d\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:43:06.027682 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:06.027677    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-model-cache\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:43:06.027873 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:06.027698    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-home\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:43:06.027873 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:06.027707    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-tmp-dir\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:43:06.027873 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:06.027715    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dd52709-2542-4c63-8aba-15dd2386f80d-dshm\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:43:06.027873 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:06.027721    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dd52709-2542-4c63-8aba-15dd2386f80d-tls-certs\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:43:06.249599 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:06.249562    2574 generic.go:356] "Generic (PLEG): container finished" podID="8dd52709-2542-4c63-8aba-15dd2386f80d" containerID="4f7ace9c2dcc88713be3a2456fca163e5b97fdc705d2cc15187dd08ff9752530" exitCode=0
Jul 28 22:43:06.249805 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:06.249613    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"
Jul 28 22:43:06.249805 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:06.249650    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs" event={"ID":"8dd52709-2542-4c63-8aba-15dd2386f80d","Type":"ContainerDied","Data":"4f7ace9c2dcc88713be3a2456fca163e5b97fdc705d2cc15187dd08ff9752530"}
Jul 28 22:43:06.249805 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:06.249701    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs" event={"ID":"8dd52709-2542-4c63-8aba-15dd2386f80d","Type":"ContainerDied","Data":"090e2b2fa5bb9deca0155f2bef9611d7b99052e7d69ea23b11100cd7dfb64736"}
Jul 28 22:43:06.249805 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:06.249724    2574 scope.go:117] "RemoveContainer" containerID="4f7ace9c2dcc88713be3a2456fca163e5b97fdc705d2cc15187dd08ff9752530"
Jul 28 22:43:06.258610 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:06.258590    2574 scope.go:117] "RemoveContainer" containerID="4f7ace9c2dcc88713be3a2456fca163e5b97fdc705d2cc15187dd08ff9752530"
Jul 28 22:43:06.258878 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:43:06.258855    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f7ace9c2dcc88713be3a2456fca163e5b97fdc705d2cc15187dd08ff9752530\": container with ID starting with 4f7ace9c2dcc88713be3a2456fca163e5b97fdc705d2cc15187dd08ff9752530 not found: ID does not exist" containerID="4f7ace9c2dcc88713be3a2456fca163e5b97fdc705d2cc15187dd08ff9752530"
Jul 28 22:43:06.258926 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:06.258888    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f7ace9c2dcc88713be3a2456fca163e5b97fdc705d2cc15187dd08ff9752530"} err="failed to get container status \"4f7ace9c2dcc88713be3a2456fca163e5b97fdc705d2cc15187dd08ff9752530\": rpc error: code = NotFound desc = could not find container \"4f7ace9c2dcc88713be3a2456fca163e5b97fdc705d2cc15187dd08ff9752530\": container with ID starting with 4f7ace9c2dcc88713be3a2456fca163e5b97fdc705d2cc15187dd08ff9752530 not found: ID does not exist"
Jul 28 22:43:06.269813 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:06.269789    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"]
Jul 28 22:43:06.273719 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:06.273698    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-00957a/test-model-00957a-kserve-7bf94b6f69-7f9gs"]
Jul 28 22:43:07.493588 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:07.493554    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dd52709-2542-4c63-8aba-15dd2386f80d" path="/var/lib/kubelet/pods/8dd52709-2542-4c63-8aba-15dd2386f80d/volumes"
Jul 28 22:43:08.985666 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:08.985636    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"]
Jul 28 22:43:08.986063 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:08.985850    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t" podUID="9ac63970-704b-4ec0-bd92-1b0230b2377c" containerName="istio-proxy" containerID="cri-o://b1ab26ede5b649bd06ec88d5e890476bf46be2b80b89978201e07e54e2a2995a" gracePeriod=30
Jul 28 22:43:14.234300 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.234276    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:43:14.292939 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.292862    2574 generic.go:356] "Generic (PLEG): container finished" podID="9ac63970-704b-4ec0-bd92-1b0230b2377c" containerID="b1ab26ede5b649bd06ec88d5e890476bf46be2b80b89978201e07e54e2a2995a" exitCode=0
Jul 28 22:43:14.292939 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.292907    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"
Jul 28 22:43:14.293119 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.292936    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t" event={"ID":"9ac63970-704b-4ec0-bd92-1b0230b2377c","Type":"ContainerDied","Data":"b1ab26ede5b649bd06ec88d5e890476bf46be2b80b89978201e07e54e2a2995a"}
Jul 28 22:43:14.293119 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.292989    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t" event={"ID":"9ac63970-704b-4ec0-bd92-1b0230b2377c","Type":"ContainerDied","Data":"4f3359ae7644122d742cce31b075d261dc5ca6fdecb30cdcb4fb9efc9c5cbbba"}
Jul 28 22:43:14.293119 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.293012    2574 scope.go:117] "RemoveContainer" containerID="b1ab26ede5b649bd06ec88d5e890476bf46be2b80b89978201e07e54e2a2995a"
Jul 28 22:43:14.301356 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.301334    2574 scope.go:117] "RemoveContainer" containerID="df7fd4b85a929f5cf04db83d1e605c13301c195178955a0f59640f9937a1fe50"
Jul 28 22:43:14.303399 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.303378    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8j9gj\" (UniqueName: \"kubernetes.io/projected/9ac63970-704b-4ec0-bd92-1b0230b2377c-kube-api-access-8j9gj\") pod \"9ac63970-704b-4ec0-bd92-1b0230b2377c\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") "
Jul 28 22:43:14.303524 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.303410    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-credential-socket\") pod \"9ac63970-704b-4ec0-bd92-1b0230b2377c\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") "
Jul 28 22:43:14.303524 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.303442    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-token\") pod \"9ac63970-704b-4ec0-bd92-1b0230b2377c\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") "
Jul 28 22:43:14.303524 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.303462    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-data\") pod \"9ac63970-704b-4ec0-bd92-1b0230b2377c\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") "
Jul 28 22:43:14.303524 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.303494    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ac63970-704b-4ec0-bd92-1b0230b2377c-istiod-ca-cert\") pod \"9ac63970-704b-4ec0-bd92-1b0230b2377c\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") "
Jul 28 22:43:14.303735 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.303528    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-workload-certs\") pod \"9ac63970-704b-4ec0-bd92-1b0230b2377c\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") "
Jul 28 22:43:14.303735 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.303554    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-envoy\") pod \"9ac63970-704b-4ec0-bd92-1b0230b2377c\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") "
Jul 28 22:43:14.303735 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.303656    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-podinfo\") pod \"9ac63970-704b-4ec0-bd92-1b0230b2377c\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") "
Jul 28 22:43:14.303735 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.303695    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-workload-socket\") pod \"9ac63970-704b-4ec0-bd92-1b0230b2377c\" (UID: \"9ac63970-704b-4ec0-bd92-1b0230b2377c\") "
Jul 28 22:43:14.303735 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.303707    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9ac63970-704b-4ec0-bd92-1b0230b2377c" (UID: "9ac63970-704b-4ec0-bd92-1b0230b2377c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:43:14.304768 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.304407    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9ac63970-704b-4ec0-bd92-1b0230b2377c" (UID: "9ac63970-704b-4ec0-bd92-1b0230b2377c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:43:14.304768 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.304733    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ac63970-704b-4ec0-bd92-1b0230b2377c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9ac63970-704b-4ec0-bd92-1b0230b2377c" (UID: "9ac63970-704b-4ec0-bd92-1b0230b2377c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:43:14.307419 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.305375    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9ac63970-704b-4ec0-bd92-1b0230b2377c" (UID: "9ac63970-704b-4ec0-bd92-1b0230b2377c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:43:14.307419 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.305786    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-workload-certs\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:43:14.307419 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.305821    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-credential-socket\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:43:14.307419 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.305834    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-token" (OuterVolumeSpecName: "istio-token") pod "9ac63970-704b-4ec0-bd92-1b0230b2377c" (UID: "9ac63970-704b-4ec0-bd92-1b0230b2377c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:43:14.307419 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.305960    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-data" (OuterVolumeSpecName: "istio-data") pod "9ac63970-704b-4ec0-bd92-1b0230b2377c" (UID: "9ac63970-704b-4ec0-bd92-1b0230b2377c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:43:14.307419 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.307307    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ac63970-704b-4ec0-bd92-1b0230b2377c-kube-api-access-8j9gj" (OuterVolumeSpecName: "kube-api-access-8j9gj") pod "9ac63970-704b-4ec0-bd92-1b0230b2377c" (UID: "9ac63970-704b-4ec0-bd92-1b0230b2377c"). InnerVolumeSpecName "kube-api-access-8j9gj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:43:14.312565 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.312534    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9ac63970-704b-4ec0-bd92-1b0230b2377c" (UID: "9ac63970-704b-4ec0-bd92-1b0230b2377c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 22:43:14.312565 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.312556    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9ac63970-704b-4ec0-bd92-1b0230b2377c" (UID: "9ac63970-704b-4ec0-bd92-1b0230b2377c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:43:14.324860 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.324842    2574 scope.go:117] "RemoveContainer" containerID="b1ab26ede5b649bd06ec88d5e890476bf46be2b80b89978201e07e54e2a2995a"
Jul 28 22:43:14.325312 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:43:14.325280    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1ab26ede5b649bd06ec88d5e890476bf46be2b80b89978201e07e54e2a2995a\": container with ID starting with b1ab26ede5b649bd06ec88d5e890476bf46be2b80b89978201e07e54e2a2995a not found: ID does not exist" containerID="b1ab26ede5b649bd06ec88d5e890476bf46be2b80b89978201e07e54e2a2995a"
Jul 28 22:43:14.325394 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.325329    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1ab26ede5b649bd06ec88d5e890476bf46be2b80b89978201e07e54e2a2995a"} err="failed to get container status \"b1ab26ede5b649bd06ec88d5e890476bf46be2b80b89978201e07e54e2a2995a\": rpc error: code = NotFound desc = could not find container \"b1ab26ede5b649bd06ec88d5e890476bf46be2b80b89978201e07e54e2a2995a\": container with ID starting with b1ab26ede5b649bd06ec88d5e890476bf46be2b80b89978201e07e54e2a2995a not found: ID does not exist"
Jul 28 22:43:14.325394 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.325377    2574 scope.go:117] "RemoveContainer" containerID="df7fd4b85a929f5cf04db83d1e605c13301c195178955a0f59640f9937a1fe50"
Jul 28 22:43:14.325646 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:43:14.325629    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df7fd4b85a929f5cf04db83d1e605c13301c195178955a0f59640f9937a1fe50\": container with ID starting with df7fd4b85a929f5cf04db83d1e605c13301c195178955a0f59640f9937a1fe50 not found: ID does not exist" containerID="df7fd4b85a929f5cf04db83d1e605c13301c195178955a0f59640f9937a1fe50"
Jul 28 22:43:14.325704 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.325651    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df7fd4b85a929f5cf04db83d1e605c13301c195178955a0f59640f9937a1fe50"} err="failed to get container status \"df7fd4b85a929f5cf04db83d1e605c13301c195178955a0f59640f9937a1fe50\": rpc error: code = NotFound desc = could not find container \"df7fd4b85a929f5cf04db83d1e605c13301c195178955a0f59640f9937a1fe50\": container with ID starting with df7fd4b85a929f5cf04db83d1e605c13301c195178955a0f59640f9937a1fe50 not found: ID does not exist"
Jul 28 22:43:14.406416 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.406385    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-token\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:43:14.406416 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.406410    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-data\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:43:14.406416 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.406420    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ac63970-704b-4ec0-bd92-1b0230b2377c-istiod-ca-cert\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:43:14.406615 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.406428    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-envoy\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:43:14.406615 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.406436    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ac63970-704b-4ec0-bd92-1b0230b2377c-istio-podinfo\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:43:14.406615 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.406443    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac63970-704b-4ec0-bd92-1b0230b2377c-workload-socket\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:43:14.406615 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.406451    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8j9gj\" (UniqueName: \"kubernetes.io/projected/9ac63970-704b-4ec0-bd92-1b0230b2377c-kube-api-access-8j9gj\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:43:14.613669 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.613641    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"]
Jul 28 22:43:14.616349 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:14.616324    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-00957a-openshift-default-5466df956b-x287t"]
Jul 28 22:43:15.493924 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:15.493888    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ac63970-704b-4ec0-bd92-1b0230b2377c" path="/var/lib/kubelet/pods/9ac63970-704b-4ec0-bd92-1b0230b2377c/volumes"
Jul 28 22:43:19.951155 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:19.951016    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l"]
Jul 28 22:43:19.951669 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:19.951279    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l" podUID="6a7c275d-20b7-4e4b-8216-a1970eb8fcd5" containerName="maas-api" containerID="cri-o://765a8c1bea88e8cc0db1ec0c3e7dbe2558a6ac198712c8d1dfcaf64419bcf0ad" gracePeriod=30
Jul 28 22:43:20.210729 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:20.210670    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l"
Jul 28 22:43:20.324507 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:20.324463    2574 generic.go:356] "Generic (PLEG): container finished" podID="6a7c275d-20b7-4e4b-8216-a1970eb8fcd5" containerID="765a8c1bea88e8cc0db1ec0c3e7dbe2558a6ac198712c8d1dfcaf64419bcf0ad" exitCode=0
Jul 28 22:43:20.324690 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:20.324522    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l"
Jul 28 22:43:20.324690 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:20.324543    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l" event={"ID":"6a7c275d-20b7-4e4b-8216-a1970eb8fcd5","Type":"ContainerDied","Data":"765a8c1bea88e8cc0db1ec0c3e7dbe2558a6ac198712c8d1dfcaf64419bcf0ad"}
Jul 28 22:43:20.324690 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:20.324598    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l" event={"ID":"6a7c275d-20b7-4e4b-8216-a1970eb8fcd5","Type":"ContainerDied","Data":"52ae95641415ca988c1df46c7f5563f62d46f72f58fb0f1faf4b732d50db1e71"}
Jul 28 22:43:20.324690 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:20.324627    2574 scope.go:117] "RemoveContainer" containerID="765a8c1bea88e8cc0db1ec0c3e7dbe2558a6ac198712c8d1dfcaf64419bcf0ad"
Jul 28 22:43:20.334499 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:20.334470    2574 scope.go:117] "RemoveContainer" containerID="765a8c1bea88e8cc0db1ec0c3e7dbe2558a6ac198712c8d1dfcaf64419bcf0ad"
Jul 28 22:43:20.334758 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:43:20.334739    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"765a8c1bea88e8cc0db1ec0c3e7dbe2558a6ac198712c8d1dfcaf64419bcf0ad\": container with ID starting with 765a8c1bea88e8cc0db1ec0c3e7dbe2558a6ac198712c8d1dfcaf64419bcf0ad not found: ID does not exist" containerID="765a8c1bea88e8cc0db1ec0c3e7dbe2558a6ac198712c8d1dfcaf64419bcf0ad"
Jul 28 22:43:20.334816 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:20.334769    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"765a8c1bea88e8cc0db1ec0c3e7dbe2558a6ac198712c8d1dfcaf64419bcf0ad"} err="failed to get container status \"765a8c1bea88e8cc0db1ec0c3e7dbe2558a6ac198712c8d1dfcaf64419bcf0ad\": rpc error: code = NotFound desc = could not find container \"765a8c1bea88e8cc0db1ec0c3e7dbe2558a6ac198712c8d1dfcaf64419bcf0ad\": container with ID starting with 765a8c1bea88e8cc0db1ec0c3e7dbe2558a6ac198712c8d1dfcaf64419bcf0ad not found: ID does not exist"
Jul 28 22:43:20.357100 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:20.357072    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a7c275d-20b7-4e4b-8216-a1970eb8fcd5-maas-api-tls\") pod \"6a7c275d-20b7-4e4b-8216-a1970eb8fcd5\" (UID: \"6a7c275d-20b7-4e4b-8216-a1970eb8fcd5\") "
Jul 28 22:43:20.357223 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:20.357106    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xknj\" (UniqueName: \"kubernetes.io/projected/6a7c275d-20b7-4e4b-8216-a1970eb8fcd5-kube-api-access-6xknj\") pod \"6a7c275d-20b7-4e4b-8216-a1970eb8fcd5\" (UID: \"6a7c275d-20b7-4e4b-8216-a1970eb8fcd5\") "
Jul 28 22:43:20.357223 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:20.357161    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a7c275d-20b7-4e4b-8216-a1970eb8fcd5-openshift-service-ca\") pod \"6a7c275d-20b7-4e4b-8216-a1970eb8fcd5\" (UID: \"6a7c275d-20b7-4e4b-8216-a1970eb8fcd5\") "
Jul 28 22:43:20.357589 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:20.357569    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a7c275d-20b7-4e4b-8216-a1970eb8fcd5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6a7c275d-20b7-4e4b-8216-a1970eb8fcd5" (UID: "6a7c275d-20b7-4e4b-8216-a1970eb8fcd5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:43:20.359109 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:20.359082    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a7c275d-20b7-4e4b-8216-a1970eb8fcd5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6a7c275d-20b7-4e4b-8216-a1970eb8fcd5" (UID: "6a7c275d-20b7-4e4b-8216-a1970eb8fcd5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:43:20.359260 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:20.359244    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a7c275d-20b7-4e4b-8216-a1970eb8fcd5-kube-api-access-6xknj" (OuterVolumeSpecName: "kube-api-access-6xknj") pod "6a7c275d-20b7-4e4b-8216-a1970eb8fcd5" (UID: "6a7c275d-20b7-4e4b-8216-a1970eb8fcd5"). InnerVolumeSpecName "kube-api-access-6xknj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:43:20.457906 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:20.457870    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a7c275d-20b7-4e4b-8216-a1970eb8fcd5-maas-api-tls\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:43:20.457906 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:20.457901    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xknj\" (UniqueName: \"kubernetes.io/projected/6a7c275d-20b7-4e4b-8216-a1970eb8fcd5-kube-api-access-6xknj\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:43:20.457906 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:20.457911    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a7c275d-20b7-4e4b-8216-a1970eb8fcd5-openshift-service-ca\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:43:20.645423 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:20.645385    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l"]
Jul 28 22:43:20.649479 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:20.649448    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8a6fb9-8dbb8d566-6n55l"]
Jul 28 22:43:21.498820 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:21.494204    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a7c275d-20b7-4e4b-8216-a1970eb8fcd5" path="/var/lib/kubelet/pods/6a7c275d-20b7-4e4b-8216-a1970eb8fcd5/volumes"
Jul 28 22:43:27.494906 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.494825    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"]
Jul 28 22:43:27.495282 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.495132    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ac63970-704b-4ec0-bd92-1b0230b2377c" containerName="istio-proxy"
Jul 28 22:43:27.495282 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.495142    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ac63970-704b-4ec0-bd92-1b0230b2377c" containerName="istio-proxy"
Jul 28 22:43:27.495282 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.495150    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ac63970-704b-4ec0-bd92-1b0230b2377c" containerName="istio-proxy"
Jul 28 22:43:27.495282 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.495155    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ac63970-704b-4ec0-bd92-1b0230b2377c" containerName="istio-proxy"
Jul 28 22:43:27.495282 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.495164    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a211f52a-eef1-41b2-9022-51d70a6f4876" containerName="maas-api"
Jul 28 22:43:27.495282 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.495170    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a211f52a-eef1-41b2-9022-51d70a6f4876" containerName="maas-api"
Jul 28 22:43:27.495282 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.495178    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dd52709-2542-4c63-8aba-15dd2386f80d" containerName="main"
Jul 28 22:43:27.495282 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.495183    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dd52709-2542-4c63-8aba-15dd2386f80d" containerName="main"
Jul 28 22:43:27.495282 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.495195    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a7c275d-20b7-4e4b-8216-a1970eb8fcd5" containerName="maas-api"
Jul 28 22:43:27.495282 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.495199    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a7c275d-20b7-4e4b-8216-a1970eb8fcd5" containerName="maas-api"
Jul 28 22:43:27.495282 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.495263    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ac63970-704b-4ec0-bd92-1b0230b2377c" containerName="istio-proxy"
Jul 28 22:43:27.495282 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.495272    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ac63970-704b-4ec0-bd92-1b0230b2377c" containerName="istio-proxy"
Jul 28 22:43:27.495282 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.495278    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a211f52a-eef1-41b2-9022-51d70a6f4876" containerName="maas-api"
Jul 28 22:43:27.495282 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.495283    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dd52709-2542-4c63-8aba-15dd2386f80d" containerName="main"
Jul 28 22:43:27.495282 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.495289    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a7c275d-20b7-4e4b-8216-a1970eb8fcd5" containerName="maas-api"
Jul 28 22:43:27.497928 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.497909    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.499899 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.499879    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-eeffd6-openshift-default-dockercfg-n4vl9\""
Jul 28 22:43:27.509038 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.509012    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"]
Jul 28 22:43:27.621085 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.621052    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-token\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.621341 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.621094    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-workload-certs\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.621341 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.621273    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-workload-socket\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.621341 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.621321    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqmkv\" (UniqueName: \"kubernetes.io/projected/562ce72a-b5ab-4c10-bc04-cda56e7e165d-kube-api-access-xqmkv\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.621527 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.621376    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-envoy\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.621527 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.621465    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-data\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.621638 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.621533    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-credential-socket\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.621638 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.621560    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-podinfo\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.621638 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.621595    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istiod-ca-cert\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.723024 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.722987    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istiod-ca-cert\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.723199 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.723046    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-token\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.723199 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.723072    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-workload-certs\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.723199 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.723114    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-workload-socket\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.723199 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.723149    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqmkv\" (UniqueName: \"kubernetes.io/projected/562ce72a-b5ab-4c10-bc04-cda56e7e165d-kube-api-access-xqmkv\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.723199 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.723182    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-envoy\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.723489 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.723226    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-data\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.723489 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.723300    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-credential-socket\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.723489 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.723329    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-podinfo\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.723642 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.723548    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-workload-certs\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.723642 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.723590    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-workload-socket\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.723751 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.723718    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-data\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.723751 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.723741    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-credential-socket\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.723878 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.723856    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istiod-ca-cert\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.725574 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.725548    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-envoy\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.725697 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.725681    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-podinfo\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.733271 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.733219    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-token\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.737198 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.737173    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqmkv\" (UniqueName: \"kubernetes.io/projected/562ce72a-b5ab-4c10-bc04-cda56e7e165d-kube-api-access-xqmkv\") pod \"e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") " pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.808276 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.808188    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:27.945047 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.944982    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"]
Jul 28 22:43:27.947268 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:43:27.947242    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod562ce72a_b5ab_4c10_bc04_cda56e7e165d.slice/crio-ab6521bfb87529f38caf2b0e36dbdd95bef3b873ac7a8ad6ed382e65d3c20bb8 WatchSource:0}: Error finding container ab6521bfb87529f38caf2b0e36dbdd95bef3b873ac7a8ad6ed382e65d3c20bb8: Status 404 returned error can't find the container with id ab6521bfb87529f38caf2b0e36dbdd95bef3b873ac7a8ad6ed382e65d3c20bb8
Jul 28 22:43:27.949322 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.949289    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:43:27.949409 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.949353    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:43:27.949409 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:27.949381    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:43:28.365057 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:28.365016    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82" event={"ID":"562ce72a-b5ab-4c10-bc04-cda56e7e165d","Type":"ContainerStarted","Data":"36187205f9f2ad75f87c63965cbadfc898de56636f4728fea2decd2559d0d77a"}
Jul 28 22:43:28.365214 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:28.365071    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82" event={"ID":"562ce72a-b5ab-4c10-bc04-cda56e7e165d","Type":"ContainerStarted","Data":"ab6521bfb87529f38caf2b0e36dbdd95bef3b873ac7a8ad6ed382e65d3c20bb8"}
Jul 28 22:43:28.383434 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:28.383384    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82" podStartSLOduration=1.383370401 podStartE2EDuration="1.383370401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:43:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:43:28.381760071 +0000 UTC m=+2337.365902689" watchObservedRunningTime="2026-07-28 22:43:28.383370401 +0000 UTC m=+2337.367513018"
Jul 28 22:43:28.808879 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:28.808838    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:28.813722 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:28.813696    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:29.369943 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:29.369908    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:29.370864 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:29.370844    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:30.363964 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:30.363930    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4"]
Jul 28 22:43:30.366663 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:30.366642    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4"
Jul 28 22:43:30.369946 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:30.369922    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-eeffd6-serving-cert\""
Jul 28 22:43:30.380296 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:30.380273    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4"]
Jul 28 22:43:30.550133 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:30.550064    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c985ff6-7d37-4aa8-b31e-50090ff1c852-openshift-service-ca\") pod \"maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4\" (UID: \"7c985ff6-7d37-4aa8-b31e-50090ff1c852\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4"
Jul 28 22:43:30.550416 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:30.550141    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phqqx\" (UniqueName: \"kubernetes.io/projected/7c985ff6-7d37-4aa8-b31e-50090ff1c852-kube-api-access-phqqx\") pod \"maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4\" (UID: \"7c985ff6-7d37-4aa8-b31e-50090ff1c852\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4"
Jul 28 22:43:30.550416 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:30.550295    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c985ff6-7d37-4aa8-b31e-50090ff1c852-maas-api-tls\") pod \"maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4\" (UID: \"7c985ff6-7d37-4aa8-b31e-50090ff1c852\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4"
Jul 28 22:43:30.651383 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:30.651292    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c985ff6-7d37-4aa8-b31e-50090ff1c852-openshift-service-ca\") pod \"maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4\" (UID: \"7c985ff6-7d37-4aa8-b31e-50090ff1c852\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4"
Jul 28 22:43:30.651383 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:30.651330    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phqqx\" (UniqueName: \"kubernetes.io/projected/7c985ff6-7d37-4aa8-b31e-50090ff1c852-kube-api-access-phqqx\") pod \"maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4\" (UID: \"7c985ff6-7d37-4aa8-b31e-50090ff1c852\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4"
Jul 28 22:43:30.651383 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:30.651377    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c985ff6-7d37-4aa8-b31e-50090ff1c852-maas-api-tls\") pod \"maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4\" (UID: \"7c985ff6-7d37-4aa8-b31e-50090ff1c852\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4"
Jul 28 22:43:30.651939 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:30.651913    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c985ff6-7d37-4aa8-b31e-50090ff1c852-openshift-service-ca\") pod \"maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4\" (UID: \"7c985ff6-7d37-4aa8-b31e-50090ff1c852\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4"
Jul 28 22:43:30.653822 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:30.653799    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c985ff6-7d37-4aa8-b31e-50090ff1c852-maas-api-tls\") pod \"maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4\" (UID: \"7c985ff6-7d37-4aa8-b31e-50090ff1c852\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4"
Jul 28 22:43:30.661516 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:30.661493    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phqqx\" (UniqueName: \"kubernetes.io/projected/7c985ff6-7d37-4aa8-b31e-50090ff1c852-kube-api-access-phqqx\") pod \"maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4\" (UID: \"7c985ff6-7d37-4aa8-b31e-50090ff1c852\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4"
Jul 28 22:43:30.678319 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:30.678298    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4"
Jul 28 22:43:30.808990 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:30.808959    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4"]
Jul 28 22:43:30.811424 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:43:30.811393    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c985ff6_7d37_4aa8_b31e_50090ff1c852.slice/crio-f3d451f325abd05b0692c8a99cfe7315bdb9a546e8f722a88148b62bf1119904 WatchSource:0}: Error finding container f3d451f325abd05b0692c8a99cfe7315bdb9a546e8f722a88148b62bf1119904: Status 404 returned error can't find the container with id f3d451f325abd05b0692c8a99cfe7315bdb9a546e8f722a88148b62bf1119904
Jul 28 22:43:31.380290 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:31.380261    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4" event={"ID":"7c985ff6-7d37-4aa8-b31e-50090ff1c852","Type":"ContainerStarted","Data":"f3d451f325abd05b0692c8a99cfe7315bdb9a546e8f722a88148b62bf1119904"}
Jul 28 22:43:32.387179 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:32.387135    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4" event={"ID":"7c985ff6-7d37-4aa8-b31e-50090ff1c852","Type":"ContainerStarted","Data":"59e3434b6d8fdd404197bb807c9aa796535b7b6148fa3b1be8f0af0196aca787"}
Jul 28 22:43:32.387663 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:32.387402    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4"
Jul 28 22:43:32.404845 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:32.404797    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4" podStartSLOduration=1.8423629400000001 podStartE2EDuration="2.404779866s" totalImagesPullingTime="562.416926ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:43:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:43:32.402849004 +0000 UTC m=+2341.386991622" watchObservedRunningTime="2026-07-28 22:43:32.404779866 +0000 UTC m=+2341.388922485"
Jul 28 22:43:38.398110 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:38.398080    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4"
Jul 28 22:43:43.441604 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:43.441512    2574 generic.go:356] "Generic (PLEG): container finished" podID="562ce72a-b5ab-4c10-bc04-cda56e7e165d" containerID="36187205f9f2ad75f87c63965cbadfc898de56636f4728fea2decd2559d0d77a" exitCode=0
Jul 28 22:43:43.442032 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:43.441599    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82" event={"ID":"562ce72a-b5ab-4c10-bc04-cda56e7e165d","Type":"ContainerDied","Data":"36187205f9f2ad75f87c63965cbadfc898de56636f4728fea2decd2559d0d77a"}
Jul 28 22:43:43.442032 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:43.442025    2574 scope.go:117] "RemoveContainer" containerID="36187205f9f2ad75f87c63965cbadfc898de56636f4728fea2decd2559d0d77a"
Jul 28 22:43:43.443088 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:43.443063    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:43:43.443155 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:43.443127    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:43:43.443191 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:43.443157    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:43:44.370047 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:44.370009    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:44.451113 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:44.451076    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82" event={"ID":"562ce72a-b5ab-4c10-bc04-cda56e7e165d","Type":"ContainerStarted","Data":"e58f9b0061fc7243c610000ffa91315d8f6b006e12ad27970122636d080d3fd0"}
Jul 28 22:43:44.808923 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:44.808888    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:44.810082 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:44.810060    2574 patch_prober.go:28] interesting pod/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 28 22:43:44.810200 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:44.810115    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82" podUID="562ce72a-b5ab-4c10-bc04-cda56e7e165d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 28 22:43:45.808822 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:45.808782    2574 patch_prober.go:28] interesting pod/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 28 22:43:45.809270 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:45.808851    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82" podUID="562ce72a-b5ab-4c10-bc04-cda56e7e165d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 28 22:43:46.809112 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:46.809075    2574 patch_prober.go:28] interesting pod/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 28 22:43:46.809567 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:46.809138    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82" podUID="562ce72a-b5ab-4c10-bc04-cda56e7e165d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 28 22:43:47.808867 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:47.808828    2574 patch_prober.go:28] interesting pod/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 28 22:43:47.809055 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:47.808901    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82" podUID="562ce72a-b5ab-4c10-bc04-cda56e7e165d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 28 22:43:48.813107 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:48.813076    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:48.813524 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:48.813412    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:48.814150 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:48.814134    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:43:58.984381 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:58.984330    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh"]
Jul 28 22:43:58.988301 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:58.988276    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh"
Jul 28 22:43:58.991055 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:58.991037    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-57f127-serving-cert\""
Jul 28 22:43:58.996905 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:58.996875    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh"]
Jul 28 22:43:59.097922 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:59.097891    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4fc5cb3-a020-4226-a9ea-cb5bce07396f-openshift-service-ca\") pod \"maas-api-e2e-api-b-57f127-fb56bb579-sj6qh\" (UID: \"e4fc5cb3-a020-4226-a9ea-cb5bce07396f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh"
Jul 28 22:43:59.098092 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:59.097943    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4fc5cb3-a020-4226-a9ea-cb5bce07396f-maas-api-tls\") pod \"maas-api-e2e-api-b-57f127-fb56bb579-sj6qh\" (UID: \"e4fc5cb3-a020-4226-a9ea-cb5bce07396f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh"
Jul 28 22:43:59.098092 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:59.098041    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqw7d\" (UniqueName: \"kubernetes.io/projected/e4fc5cb3-a020-4226-a9ea-cb5bce07396f-kube-api-access-bqw7d\") pod \"maas-api-e2e-api-b-57f127-fb56bb579-sj6qh\" (UID: \"e4fc5cb3-a020-4226-a9ea-cb5bce07396f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh"
Jul 28 22:43:59.198968 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:59.198931    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4fc5cb3-a020-4226-a9ea-cb5bce07396f-maas-api-tls\") pod \"maas-api-e2e-api-b-57f127-fb56bb579-sj6qh\" (UID: \"e4fc5cb3-a020-4226-a9ea-cb5bce07396f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh"
Jul 28 22:43:59.199159 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:59.199022    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqw7d\" (UniqueName: \"kubernetes.io/projected/e4fc5cb3-a020-4226-a9ea-cb5bce07396f-kube-api-access-bqw7d\") pod \"maas-api-e2e-api-b-57f127-fb56bb579-sj6qh\" (UID: \"e4fc5cb3-a020-4226-a9ea-cb5bce07396f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh"
Jul 28 22:43:59.199159 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:59.199082    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4fc5cb3-a020-4226-a9ea-cb5bce07396f-openshift-service-ca\") pod \"maas-api-e2e-api-b-57f127-fb56bb579-sj6qh\" (UID: \"e4fc5cb3-a020-4226-a9ea-cb5bce07396f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh"
Jul 28 22:43:59.199724 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:59.199699    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4fc5cb3-a020-4226-a9ea-cb5bce07396f-openshift-service-ca\") pod \"maas-api-e2e-api-b-57f127-fb56bb579-sj6qh\" (UID: \"e4fc5cb3-a020-4226-a9ea-cb5bce07396f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh"
Jul 28 22:43:59.201352 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:59.201325    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4fc5cb3-a020-4226-a9ea-cb5bce07396f-maas-api-tls\") pod \"maas-api-e2e-api-b-57f127-fb56bb579-sj6qh\" (UID: \"e4fc5cb3-a020-4226-a9ea-cb5bce07396f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh"
Jul 28 22:43:59.206277 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:59.206257    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqw7d\" (UniqueName: \"kubernetes.io/projected/e4fc5cb3-a020-4226-a9ea-cb5bce07396f-kube-api-access-bqw7d\") pod \"maas-api-e2e-api-b-57f127-fb56bb579-sj6qh\" (UID: \"e4fc5cb3-a020-4226-a9ea-cb5bce07396f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh"
Jul 28 22:43:59.301433 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:59.301344    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh"
Jul 28 22:43:59.437726 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:59.437695    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh"]
Jul 28 22:43:59.440256 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:43:59.440216    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4fc5cb3_a020_4226_a9ea_cb5bce07396f.slice/crio-c2e60769a0f0dcefc74f29754dae3487574646265b5a4eab37707155166c6676 WatchSource:0}: Error finding container c2e60769a0f0dcefc74f29754dae3487574646265b5a4eab37707155166c6676: Status 404 returned error can't find the container with id c2e60769a0f0dcefc74f29754dae3487574646265b5a4eab37707155166c6676
Jul 28 22:43:59.523924 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:43:59.523888    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh" event={"ID":"e4fc5cb3-a020-4226-a9ea-cb5bce07396f","Type":"ContainerStarted","Data":"c2e60769a0f0dcefc74f29754dae3487574646265b5a4eab37707155166c6676"}
Jul 28 22:44:00.530150 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:00.530114    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh" event={"ID":"e4fc5cb3-a020-4226-a9ea-cb5bce07396f","Type":"ContainerStarted","Data":"04a72f1672e7f416a5351eb1ebb6fce8b3a3d85b7059a118c5dcaafb49458bce"}
Jul 28 22:44:00.530599 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:00.530220    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh"
Jul 28 22:44:00.547242 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:00.547190    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh" podStartSLOduration=2.18681589 podStartE2EDuration="2.547175915s" totalImagesPullingTime="360.360025ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:43:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:44:00.545455541 +0000 UTC m=+2369.529598164" watchObservedRunningTime="2026-07-28 22:44:00.547175915 +0000 UTC m=+2369.531318577"
Jul 28 22:44:06.540126 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:06.540096    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh"
Jul 28 22:44:42.819717 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:42.819660    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4"]
Jul 28 22:44:42.820317 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:42.819915    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4" podUID="7c985ff6-7d37-4aa8-b31e-50090ff1c852" containerName="maas-api" containerID="cri-o://59e3434b6d8fdd404197bb807c9aa796535b7b6148fa3b1be8f0af0196aca787" gracePeriod=30
Jul 28 22:44:43.078088 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:43.078031    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4"
Jul 28 22:44:43.183566 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:43.183531    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c985ff6-7d37-4aa8-b31e-50090ff1c852-maas-api-tls\") pod \"7c985ff6-7d37-4aa8-b31e-50090ff1c852\" (UID: \"7c985ff6-7d37-4aa8-b31e-50090ff1c852\") "
Jul 28 22:44:43.183749 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:43.183584    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phqqx\" (UniqueName: \"kubernetes.io/projected/7c985ff6-7d37-4aa8-b31e-50090ff1c852-kube-api-access-phqqx\") pod \"7c985ff6-7d37-4aa8-b31e-50090ff1c852\" (UID: \"7c985ff6-7d37-4aa8-b31e-50090ff1c852\") "
Jul 28 22:44:43.183749 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:43.183636    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c985ff6-7d37-4aa8-b31e-50090ff1c852-openshift-service-ca\") pod \"7c985ff6-7d37-4aa8-b31e-50090ff1c852\" (UID: \"7c985ff6-7d37-4aa8-b31e-50090ff1c852\") "
Jul 28 22:44:43.184073 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:43.184040    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c985ff6-7d37-4aa8-b31e-50090ff1c852-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7c985ff6-7d37-4aa8-b31e-50090ff1c852" (UID: "7c985ff6-7d37-4aa8-b31e-50090ff1c852"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:44:43.185836 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:43.185806    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c985ff6-7d37-4aa8-b31e-50090ff1c852-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7c985ff6-7d37-4aa8-b31e-50090ff1c852" (UID: "7c985ff6-7d37-4aa8-b31e-50090ff1c852"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:44:43.185836 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:43.185813    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c985ff6-7d37-4aa8-b31e-50090ff1c852-kube-api-access-phqqx" (OuterVolumeSpecName: "kube-api-access-phqqx") pod "7c985ff6-7d37-4aa8-b31e-50090ff1c852" (UID: "7c985ff6-7d37-4aa8-b31e-50090ff1c852"). InnerVolumeSpecName "kube-api-access-phqqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:44:43.285008 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:43.284969    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c985ff6-7d37-4aa8-b31e-50090ff1c852-maas-api-tls\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:44:43.285008 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:43.285003    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phqqx\" (UniqueName: \"kubernetes.io/projected/7c985ff6-7d37-4aa8-b31e-50090ff1c852-kube-api-access-phqqx\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:44:43.285008 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:43.285012    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c985ff6-7d37-4aa8-b31e-50090ff1c852-openshift-service-ca\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:44:43.734905 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:43.734865    2574 generic.go:356] "Generic (PLEG): container finished" podID="7c985ff6-7d37-4aa8-b31e-50090ff1c852" containerID="59e3434b6d8fdd404197bb807c9aa796535b7b6148fa3b1be8f0af0196aca787" exitCode=0
Jul 28 22:44:43.734905 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:43.734903    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4"
Jul 28 22:44:43.735157 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:43.734979    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4" event={"ID":"7c985ff6-7d37-4aa8-b31e-50090ff1c852","Type":"ContainerDied","Data":"59e3434b6d8fdd404197bb807c9aa796535b7b6148fa3b1be8f0af0196aca787"}
Jul 28 22:44:43.735157 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:43.735031    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4" event={"ID":"7c985ff6-7d37-4aa8-b31e-50090ff1c852","Type":"ContainerDied","Data":"f3d451f325abd05b0692c8a99cfe7315bdb9a546e8f722a88148b62bf1119904"}
Jul 28 22:44:43.735157 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:43.735052    2574 scope.go:117] "RemoveContainer" containerID="59e3434b6d8fdd404197bb807c9aa796535b7b6148fa3b1be8f0af0196aca787"
Jul 28 22:44:43.745501 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:43.745482    2574 scope.go:117] "RemoveContainer" containerID="59e3434b6d8fdd404197bb807c9aa796535b7b6148fa3b1be8f0af0196aca787"
Jul 28 22:44:43.745820 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:44:43.745803    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59e3434b6d8fdd404197bb807c9aa796535b7b6148fa3b1be8f0af0196aca787\": container with ID starting with 59e3434b6d8fdd404197bb807c9aa796535b7b6148fa3b1be8f0af0196aca787 not found: ID does not exist" containerID="59e3434b6d8fdd404197bb807c9aa796535b7b6148fa3b1be8f0af0196aca787"
Jul 28 22:44:43.745878 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:43.745827    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59e3434b6d8fdd404197bb807c9aa796535b7b6148fa3b1be8f0af0196aca787"} err="failed to get container status \"59e3434b6d8fdd404197bb807c9aa796535b7b6148fa3b1be8f0af0196aca787\": rpc error: code = NotFound desc = could not find container \"59e3434b6d8fdd404197bb807c9aa796535b7b6148fa3b1be8f0af0196aca787\": container with ID starting with 59e3434b6d8fdd404197bb807c9aa796535b7b6148fa3b1be8f0af0196aca787 not found: ID does not exist"
Jul 28 22:44:43.750269 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:43.750242    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4"]
Jul 28 22:44:43.753875 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:43.753815    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eeffd6-74cfbd555d-w5wv4"]
Jul 28 22:44:45.493848 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:45.493806    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c985ff6-7d37-4aa8-b31e-50090ff1c852" path="/var/lib/kubelet/pods/7c985ff6-7d37-4aa8-b31e-50090ff1c852/volumes"
Jul 28 22:44:49.440860 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:49.440823    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"]
Jul 28 22:44:49.441364 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:49.441145    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82" podUID="562ce72a-b5ab-4c10-bc04-cda56e7e165d" containerName="istio-proxy" containerID="cri-o://e58f9b0061fc7243c610000ffa91315d8f6b006e12ad27970122636d080d3fd0" gracePeriod=30
Jul 28 22:44:54.698892 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.698868    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:44:54.793543 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.793449    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-credential-socket\") pod \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") "
Jul 28 22:44:54.793543 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.793513    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-data\") pod \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") "
Jul 28 22:44:54.793543 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.793534    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-podinfo\") pod \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") "
Jul 28 22:44:54.793816 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.793560    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istiod-ca-cert\") pod \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") "
Jul 28 22:44:54.793816 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.793591    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-workload-socket\") pod \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") "
Jul 28 22:44:54.793816 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.793618    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-token\") pod \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") "
Jul 28 22:44:54.793816 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.793676    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-workload-certs\") pod \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") "
Jul 28 22:44:54.793816 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.793718    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-envoy\") pod \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") "
Jul 28 22:44:54.793816 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.793778    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqmkv\" (UniqueName: \"kubernetes.io/projected/562ce72a-b5ab-4c10-bc04-cda56e7e165d-kube-api-access-xqmkv\") pod \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\" (UID: \"562ce72a-b5ab-4c10-bc04-cda56e7e165d\") "
Jul 28 22:44:54.794144 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.793771    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "562ce72a-b5ab-4c10-bc04-cda56e7e165d" (UID: "562ce72a-b5ab-4c10-bc04-cda56e7e165d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:44:54.794144 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.793987    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "562ce72a-b5ab-4c10-bc04-cda56e7e165d" (UID: "562ce72a-b5ab-4c10-bc04-cda56e7e165d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:44:54.794144 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.794078    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "562ce72a-b5ab-4c10-bc04-cda56e7e165d" (UID: "562ce72a-b5ab-4c10-bc04-cda56e7e165d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:44:54.794411 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.794390    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "562ce72a-b5ab-4c10-bc04-cda56e7e165d" (UID: "562ce72a-b5ab-4c10-bc04-cda56e7e165d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:44:54.794553 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.794535    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-data" (OuterVolumeSpecName: "istio-data") pod "562ce72a-b5ab-4c10-bc04-cda56e7e165d" (UID: "562ce72a-b5ab-4c10-bc04-cda56e7e165d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:44:54.796350 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.796321    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "562ce72a-b5ab-4c10-bc04-cda56e7e165d" (UID: "562ce72a-b5ab-4c10-bc04-cda56e7e165d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:44:54.796350 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.796320    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-token" (OuterVolumeSpecName: "istio-token") pod "562ce72a-b5ab-4c10-bc04-cda56e7e165d" (UID: "562ce72a-b5ab-4c10-bc04-cda56e7e165d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:44:54.796509 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.796463    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "562ce72a-b5ab-4c10-bc04-cda56e7e165d" (UID: "562ce72a-b5ab-4c10-bc04-cda56e7e165d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 22:44:54.796509 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.796462    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/562ce72a-b5ab-4c10-bc04-cda56e7e165d-kube-api-access-xqmkv" (OuterVolumeSpecName: "kube-api-access-xqmkv") pod "562ce72a-b5ab-4c10-bc04-cda56e7e165d" (UID: "562ce72a-b5ab-4c10-bc04-cda56e7e165d"). InnerVolumeSpecName "kube-api-access-xqmkv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:44:54.800211 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.800189    2574 generic.go:356] "Generic (PLEG): container finished" podID="562ce72a-b5ab-4c10-bc04-cda56e7e165d" containerID="e58f9b0061fc7243c610000ffa91315d8f6b006e12ad27970122636d080d3fd0" exitCode=0
Jul 28 22:44:54.800341 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.800266    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"
Jul 28 22:44:54.800341 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.800265    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82" event={"ID":"562ce72a-b5ab-4c10-bc04-cda56e7e165d","Type":"ContainerDied","Data":"e58f9b0061fc7243c610000ffa91315d8f6b006e12ad27970122636d080d3fd0"}
Jul 28 22:44:54.800341 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.800323    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82" event={"ID":"562ce72a-b5ab-4c10-bc04-cda56e7e165d","Type":"ContainerDied","Data":"ab6521bfb87529f38caf2b0e36dbdd95bef3b873ac7a8ad6ed382e65d3c20bb8"}
Jul 28 22:44:54.800442 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.800346    2574 scope.go:117] "RemoveContainer" containerID="e58f9b0061fc7243c610000ffa91315d8f6b006e12ad27970122636d080d3fd0"
Jul 28 22:44:54.817108 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.817090    2574 scope.go:117] "RemoveContainer" containerID="36187205f9f2ad75f87c63965cbadfc898de56636f4728fea2decd2559d0d77a"
Jul 28 22:44:54.824817 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.824801    2574 scope.go:117] "RemoveContainer" containerID="e58f9b0061fc7243c610000ffa91315d8f6b006e12ad27970122636d080d3fd0"
Jul 28 22:44:54.825428 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:44:54.825290    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e58f9b0061fc7243c610000ffa91315d8f6b006e12ad27970122636d080d3fd0\": container with ID starting with e58f9b0061fc7243c610000ffa91315d8f6b006e12ad27970122636d080d3fd0 not found: ID does not exist" containerID="e58f9b0061fc7243c610000ffa91315d8f6b006e12ad27970122636d080d3fd0"
Jul 28 22:44:54.825428 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.825342    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e58f9b0061fc7243c610000ffa91315d8f6b006e12ad27970122636d080d3fd0"} err="failed to get container status \"e58f9b0061fc7243c610000ffa91315d8f6b006e12ad27970122636d080d3fd0\": rpc error: code = NotFound desc = could not find container \"e58f9b0061fc7243c610000ffa91315d8f6b006e12ad27970122636d080d3fd0\": container with ID starting with e58f9b0061fc7243c610000ffa91315d8f6b006e12ad27970122636d080d3fd0 not found: ID does not exist"
Jul 28 22:44:54.825428 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.825366    2574 scope.go:117] "RemoveContainer" containerID="36187205f9f2ad75f87c63965cbadfc898de56636f4728fea2decd2559d0d77a"
Jul 28 22:44:54.825697 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:44:54.825681    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36187205f9f2ad75f87c63965cbadfc898de56636f4728fea2decd2559d0d77a\": container with ID starting with 36187205f9f2ad75f87c63965cbadfc898de56636f4728fea2decd2559d0d77a not found: ID does not exist" containerID="36187205f9f2ad75f87c63965cbadfc898de56636f4728fea2decd2559d0d77a"
Jul 28 22:44:54.825753 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.825702    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36187205f9f2ad75f87c63965cbadfc898de56636f4728fea2decd2559d0d77a"} err="failed to get container status \"36187205f9f2ad75f87c63965cbadfc898de56636f4728fea2decd2559d0d77a\": rpc error: code = NotFound desc = could not find container \"36187205f9f2ad75f87c63965cbadfc898de56636f4728fea2decd2559d0d77a\": container with ID starting with 36187205f9f2ad75f87c63965cbadfc898de56636f4728fea2decd2559d0d77a not found: ID does not exist"
Jul 28 22:44:54.827348 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.827328    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"]
Jul 28 22:44:54.830902 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.830881    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-eeffd6-openshift-default-b76b97996-8dl82"]
Jul 28 22:44:54.894758 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.894728    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-envoy\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:44:54.894758 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.894756    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqmkv\" (UniqueName: \"kubernetes.io/projected/562ce72a-b5ab-4c10-bc04-cda56e7e165d-kube-api-access-xqmkv\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:44:54.894943 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.894767    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-credential-socket\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:44:54.894943 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.894776    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-data\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:44:54.894943 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.894784    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-podinfo\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:44:54.894943 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.894792    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istiod-ca-cert\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:44:54.894943 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.894800    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-workload-socket\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:44:54.894943 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.894808    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/562ce72a-b5ab-4c10-bc04-cda56e7e165d-istio-token\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:44:54.894943 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:54.894816    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/562ce72a-b5ab-4c10-bc04-cda56e7e165d-workload-certs\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:44:55.494211 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:44:55.494182    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="562ce72a-b5ab-4c10-bc04-cda56e7e165d" path="/var/lib/kubelet/pods/562ce72a-b5ab-4c10-bc04-cda56e7e165d/volumes"
Jul 28 22:45:05.258708 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:05.258670    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh"]
Jul 28 22:45:05.259493 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:05.259446    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh" podUID="e4fc5cb3-a020-4226-a9ea-cb5bce07396f" containerName="maas-api" containerID="cri-o://04a72f1672e7f416a5351eb1ebb6fce8b3a3d85b7059a118c5dcaafb49458bce" gracePeriod=30
Jul 28 22:45:05.509185 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:05.509123    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh"
Jul 28 22:45:05.587240 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:05.587198    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4fc5cb3-a020-4226-a9ea-cb5bce07396f-maas-api-tls\") pod \"e4fc5cb3-a020-4226-a9ea-cb5bce07396f\" (UID: \"e4fc5cb3-a020-4226-a9ea-cb5bce07396f\") "
Jul 28 22:45:05.587419 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:05.587260    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4fc5cb3-a020-4226-a9ea-cb5bce07396f-openshift-service-ca\") pod \"e4fc5cb3-a020-4226-a9ea-cb5bce07396f\" (UID: \"e4fc5cb3-a020-4226-a9ea-cb5bce07396f\") "
Jul 28 22:45:05.587419 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:05.587328    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqw7d\" (UniqueName: \"kubernetes.io/projected/e4fc5cb3-a020-4226-a9ea-cb5bce07396f-kube-api-access-bqw7d\") pod \"e4fc5cb3-a020-4226-a9ea-cb5bce07396f\" (UID: \"e4fc5cb3-a020-4226-a9ea-cb5bce07396f\") "
Jul 28 22:45:05.587586 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:05.587561    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4fc5cb3-a020-4226-a9ea-cb5bce07396f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e4fc5cb3-a020-4226-a9ea-cb5bce07396f" (UID: "e4fc5cb3-a020-4226-a9ea-cb5bce07396f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:45:05.589335 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:05.589312    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4fc5cb3-a020-4226-a9ea-cb5bce07396f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e4fc5cb3-a020-4226-a9ea-cb5bce07396f" (UID: "e4fc5cb3-a020-4226-a9ea-cb5bce07396f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:45:05.589429 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:05.589337    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4fc5cb3-a020-4226-a9ea-cb5bce07396f-kube-api-access-bqw7d" (OuterVolumeSpecName: "kube-api-access-bqw7d") pod "e4fc5cb3-a020-4226-a9ea-cb5bce07396f" (UID: "e4fc5cb3-a020-4226-a9ea-cb5bce07396f"). InnerVolumeSpecName "kube-api-access-bqw7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:45:05.687892 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:05.687858    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4fc5cb3-a020-4226-a9ea-cb5bce07396f-maas-api-tls\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:45:05.687892 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:05.687888    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4fc5cb3-a020-4226-a9ea-cb5bce07396f-openshift-service-ca\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:45:05.687892 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:05.687899    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqw7d\" (UniqueName: \"kubernetes.io/projected/e4fc5cb3-a020-4226-a9ea-cb5bce07396f-kube-api-access-bqw7d\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:45:05.856858 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:05.856780    2574 generic.go:356] "Generic (PLEG): container finished" podID="e4fc5cb3-a020-4226-a9ea-cb5bce07396f" containerID="04a72f1672e7f416a5351eb1ebb6fce8b3a3d85b7059a118c5dcaafb49458bce" exitCode=0
Jul 28 22:45:05.856858 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:05.856822    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh"
Jul 28 22:45:05.857026 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:05.856866    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh" event={"ID":"e4fc5cb3-a020-4226-a9ea-cb5bce07396f","Type":"ContainerDied","Data":"04a72f1672e7f416a5351eb1ebb6fce8b3a3d85b7059a118c5dcaafb49458bce"}
Jul 28 22:45:05.857026 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:05.856937    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh" event={"ID":"e4fc5cb3-a020-4226-a9ea-cb5bce07396f","Type":"ContainerDied","Data":"c2e60769a0f0dcefc74f29754dae3487574646265b5a4eab37707155166c6676"}
Jul 28 22:45:05.857026 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:05.856959    2574 scope.go:117] "RemoveContainer" containerID="04a72f1672e7f416a5351eb1ebb6fce8b3a3d85b7059a118c5dcaafb49458bce"
Jul 28 22:45:05.865367 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:05.865352    2574 scope.go:117] "RemoveContainer" containerID="04a72f1672e7f416a5351eb1ebb6fce8b3a3d85b7059a118c5dcaafb49458bce"
Jul 28 22:45:05.865619 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:45:05.865600    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04a72f1672e7f416a5351eb1ebb6fce8b3a3d85b7059a118c5dcaafb49458bce\": container with ID starting with 04a72f1672e7f416a5351eb1ebb6fce8b3a3d85b7059a118c5dcaafb49458bce not found: ID does not exist" containerID="04a72f1672e7f416a5351eb1ebb6fce8b3a3d85b7059a118c5dcaafb49458bce"
Jul 28 22:45:05.865678 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:05.865628    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04a72f1672e7f416a5351eb1ebb6fce8b3a3d85b7059a118c5dcaafb49458bce"} err="failed to get container status \"04a72f1672e7f416a5351eb1ebb6fce8b3a3d85b7059a118c5dcaafb49458bce\": rpc error: code = NotFound desc = could not find container \"04a72f1672e7f416a5351eb1ebb6fce8b3a3d85b7059a118c5dcaafb49458bce\": container with ID starting with 04a72f1672e7f416a5351eb1ebb6fce8b3a3d85b7059a118c5dcaafb49458bce not found: ID does not exist"
Jul 28 22:45:05.876168 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:05.876146    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh"]
Jul 28 22:45:05.879789 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:05.879772    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-57f127-fb56bb579-sj6qh"]
Jul 28 22:45:07.493374 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:07.493345    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4fc5cb3-a020-4226-a9ea-cb5bce07396f" path="/var/lib/kubelet/pods/e4fc5cb3-a020-4226-a9ea-cb5bce07396f/volumes"
Jul 28 22:45:13.271705 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.271667    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"]
Jul 28 22:45:13.272214 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.272194    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4fc5cb3-a020-4226-a9ea-cb5bce07396f" containerName="maas-api"
Jul 28 22:45:13.272318 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.272218    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4fc5cb3-a020-4226-a9ea-cb5bce07396f" containerName="maas-api"
Jul 28 22:45:13.272318 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.272250    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c985ff6-7d37-4aa8-b31e-50090ff1c852" containerName="maas-api"
Jul 28 22:45:13.272318 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.272258    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c985ff6-7d37-4aa8-b31e-50090ff1c852" containerName="maas-api"
Jul 28 22:45:13.272318 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.272267    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562ce72a-b5ab-4c10-bc04-cda56e7e165d" containerName="istio-proxy"
Jul 28 22:45:13.272318 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.272275    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="562ce72a-b5ab-4c10-bc04-cda56e7e165d" containerName="istio-proxy"
Jul 28 22:45:13.272318 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.272289    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562ce72a-b5ab-4c10-bc04-cda56e7e165d" containerName="istio-proxy"
Jul 28 22:45:13.272318 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.272297    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="562ce72a-b5ab-4c10-bc04-cda56e7e165d" containerName="istio-proxy"
Jul 28 22:45:13.272664 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.272410    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c985ff6-7d37-4aa8-b31e-50090ff1c852" containerName="maas-api"
Jul 28 22:45:13.272664 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.272424    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="562ce72a-b5ab-4c10-bc04-cda56e7e165d" containerName="istio-proxy"
Jul 28 22:45:13.272664 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.272433    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4fc5cb3-a020-4226-a9ea-cb5bce07396f" containerName="maas-api"
Jul 28 22:45:13.272664 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.272443    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="562ce72a-b5ab-4c10-bc04-cda56e7e165d" containerName="istio-proxy"
Jul 28 22:45:13.274731 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.274708    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.276973 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.276954    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-ae4fa9-openshift-default-dockercfg-pvlnv\""
Jul 28 22:45:13.288280 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.288217    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"]
Jul 28 22:45:13.356667 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.356632    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-credential-socket\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.356818 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.356680    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-podinfo\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.356818 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.356758    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c474c0ba-3644-4872-9921-4bb7d49856a8-istiod-ca-cert\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.356818 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.356806    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-token\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.356951 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.356825    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-envoy\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.356951 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.356876    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-workload-socket\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.356951 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.356895    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-data\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.356951 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.356915    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-workload-certs\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.356951 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.356931    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8g8p\" (UniqueName: \"kubernetes.io/projected/c474c0ba-3644-4872-9921-4bb7d49856a8-kube-api-access-r8g8p\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.457866 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.457833    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-credential-socket\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.458050 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.457882    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-podinfo\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.458050 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.457927    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c474c0ba-3644-4872-9921-4bb7d49856a8-istiod-ca-cert\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.458050 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.457967    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-token\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.458050 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.457991    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-envoy\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.458268 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.458143    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-workload-socket\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.458268 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.458189    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-data\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.458268 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.458224    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-workload-certs\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.458439 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.458278    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8g8p\" (UniqueName: \"kubernetes.io/projected/c474c0ba-3644-4872-9921-4bb7d49856a8-kube-api-access-r8g8p\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.458439 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.458277    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-credential-socket\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.458559 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.458506    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-workload-socket\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.458623 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.458609    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-data\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.458692 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.458672    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-workload-certs\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.458740 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.458696    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c474c0ba-3644-4872-9921-4bb7d49856a8-istiod-ca-cert\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.460725 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.460705    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-envoy\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.460891 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.460874    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-podinfo\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.465356 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.465331    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-token\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.465633 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.465614    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8g8p\" (UniqueName: \"kubernetes.io/projected/c474c0ba-3644-4872-9921-4bb7d49856a8-kube-api-access-r8g8p\") pod \"e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") " pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.585056 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.584978    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:13.718995 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.718970    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"]
Jul 28 22:45:13.720608 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:45:13.720582    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc474c0ba_3644_4872_9921_4bb7d49856a8.slice/crio-15216259c1ab8d9b1b4a4f6815eb378686715abde807a1127477a715c74e7290 WatchSource:0}: Error finding container 15216259c1ab8d9b1b4a4f6815eb378686715abde807a1127477a715c74e7290: Status 404 returned error can't find the container with id 15216259c1ab8d9b1b4a4f6815eb378686715abde807a1127477a715c74e7290
Jul 28 22:45:13.722655 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.722625    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:45:13.722726 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.722685    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:45:13.722726 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.722713    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:45:13.896310 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.896192    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj" event={"ID":"c474c0ba-3644-4872-9921-4bb7d49856a8","Type":"ContainerStarted","Data":"f18ae5ac9d5b8f41f1331e12bdc28b95107841c6dff4acb91d7c03b68f831312"}
Jul 28 22:45:13.896858 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.896293    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj" event={"ID":"c474c0ba-3644-4872-9921-4bb7d49856a8","Type":"ContainerStarted","Data":"15216259c1ab8d9b1b4a4f6815eb378686715abde807a1127477a715c74e7290"}
Jul 28 22:45:13.915805 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:13.915756    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj" podStartSLOduration=0.915740679 podStartE2EDuration="915.740679ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:45:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:45:13.913570902 +0000 UTC m=+2442.897713521" watchObservedRunningTime="2026-07-28 22:45:13.915740679 +0000 UTC m=+2442.899883298"
Jul 28 22:45:14.585477 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:14.585437    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:14.589916 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:14.589889    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:14.901789 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:14.901705    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:14.902495 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:14.902474    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:16.004158 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:16.004123    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf"]
Jul 28 22:45:16.006565 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:16.006544    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf"
Jul 28 22:45:16.008391 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:16.008371    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-ae4fa9-serving-cert\""
Jul 28 22:45:16.018105 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:16.018077    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf"]
Jul 28 22:45:16.182779 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:16.182742    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgwrh\" (UniqueName: \"kubernetes.io/projected/b86255cd-4ef7-474b-a335-a581d64b7cda-kube-api-access-vgwrh\") pod \"maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf\" (UID: \"b86255cd-4ef7-474b-a335-a581d64b7cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf"
Jul 28 22:45:16.182936 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:16.182797    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b86255cd-4ef7-474b-a335-a581d64b7cda-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf\" (UID: \"b86255cd-4ef7-474b-a335-a581d64b7cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf"
Jul 28 22:45:16.182936 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:16.182879    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b86255cd-4ef7-474b-a335-a581d64b7cda-maas-api-tls\") pod \"maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf\" (UID: \"b86255cd-4ef7-474b-a335-a581d64b7cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf"
Jul 28 22:45:16.283918 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:16.283837    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgwrh\" (UniqueName: \"kubernetes.io/projected/b86255cd-4ef7-474b-a335-a581d64b7cda-kube-api-access-vgwrh\") pod \"maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf\" (UID: \"b86255cd-4ef7-474b-a335-a581d64b7cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf"
Jul 28 22:45:16.283918 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:16.283883    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b86255cd-4ef7-474b-a335-a581d64b7cda-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf\" (UID: \"b86255cd-4ef7-474b-a335-a581d64b7cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf"
Jul 28 22:45:16.284105 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:16.284003    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b86255cd-4ef7-474b-a335-a581d64b7cda-maas-api-tls\") pod \"maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf\" (UID: \"b86255cd-4ef7-474b-a335-a581d64b7cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf"
Jul 28 22:45:16.284516 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:16.284498    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b86255cd-4ef7-474b-a335-a581d64b7cda-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf\" (UID: \"b86255cd-4ef7-474b-a335-a581d64b7cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf"
Jul 28 22:45:16.286324 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:16.286306    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b86255cd-4ef7-474b-a335-a581d64b7cda-maas-api-tls\") pod \"maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf\" (UID: \"b86255cd-4ef7-474b-a335-a581d64b7cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf"
Jul 28 22:45:16.291047 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:16.291031    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgwrh\" (UniqueName: \"kubernetes.io/projected/b86255cd-4ef7-474b-a335-a581d64b7cda-kube-api-access-vgwrh\") pod \"maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf\" (UID: \"b86255cd-4ef7-474b-a335-a581d64b7cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf"
Jul 28 22:45:16.316833 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:16.316803    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf"
Jul 28 22:45:16.476819 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:16.476793    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf"]
Jul 28 22:45:16.478979 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:45:16.478950    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb86255cd_4ef7_474b_a335_a581d64b7cda.slice/crio-3d4f3f8b332d2fe516a84046335f7f3ad00b9e9fd93ddd2f99886441ef2db236 WatchSource:0}: Error finding container 3d4f3f8b332d2fe516a84046335f7f3ad00b9e9fd93ddd2f99886441ef2db236: Status 404 returned error can't find the container with id 3d4f3f8b332d2fe516a84046335f7f3ad00b9e9fd93ddd2f99886441ef2db236
Jul 28 22:45:16.914321 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:16.914283    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf" event={"ID":"b86255cd-4ef7-474b-a335-a581d64b7cda","Type":"ContainerStarted","Data":"3d4f3f8b332d2fe516a84046335f7f3ad00b9e9fd93ddd2f99886441ef2db236"}
Jul 28 22:45:17.920406 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:17.920370    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf" event={"ID":"b86255cd-4ef7-474b-a335-a581d64b7cda","Type":"ContainerStarted","Data":"6e8fee90f528177ecd57c21bee41a03c6e7dde2272b8ec6ea33d54a01de34002"}
Jul 28 22:45:17.920778 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:17.920486    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf"
Jul 28 22:45:17.937474 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:17.937419    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf" podStartSLOduration=2.427263226 podStartE2EDuration="2.937399994s" totalImagesPullingTime="510.136768ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:45:17.934322206 +0000 UTC m=+2446.918464824" watchObservedRunningTime="2026-07-28 22:45:17.937399994 +0000 UTC m=+2446.921542614"
Jul 28 22:45:23.930825 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:23.930793    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf"
Jul 28 22:45:29.902642 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:29.902607    2574 patch_prober.go:28] interesting pod/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Jul 28 22:45:29.903064 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:29.902661    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj" podUID="c474c0ba-3644-4872-9921-4bb7d49856a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Jul 28 22:45:29.986451 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:29.986415    2574 generic.go:356] "Generic (PLEG): container finished" podID="c474c0ba-3644-4872-9921-4bb7d49856a8" containerID="f18ae5ac9d5b8f41f1331e12bdc28b95107841c6dff4acb91d7c03b68f831312" exitCode=0
Jul 28 22:45:29.986647 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:29.986455    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj" event={"ID":"c474c0ba-3644-4872-9921-4bb7d49856a8","Type":"ContainerDied","Data":"f18ae5ac9d5b8f41f1331e12bdc28b95107841c6dff4acb91d7c03b68f831312"}
Jul 28 22:45:29.986896 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:29.986877    2574 scope.go:117] "RemoveContainer" containerID="f18ae5ac9d5b8f41f1331e12bdc28b95107841c6dff4acb91d7c03b68f831312"
Jul 28 22:45:29.987904 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:29.987873    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:45:29.988004 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:29.987945    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:45:29.988004 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:29.987982    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:45:30.992648 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:30.992613    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj" event={"ID":"c474c0ba-3644-4872-9921-4bb7d49856a8","Type":"ContainerStarted","Data":"004dd808bd5e27cd52246245c3c76ec4990508c3837d29d4daba073a98fdfb13"}
Jul 28 22:45:31.585440 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:31.585399    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:31.585643 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:31.585617    2574 patch_prober.go:28] interesting pod/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Jul 28 22:45:31.585690 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:31.585673    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj" podUID="c474c0ba-3644-4872-9921-4bb7d49856a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Jul 28 22:45:32.585570 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:32.585533    2574 patch_prober.go:28] interesting pod/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Jul 28 22:45:32.586033 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:32.585589    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj" podUID="c474c0ba-3644-4872-9921-4bb7d49856a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Jul 28 22:45:33.585982 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:33.585937    2574 patch_prober.go:28] interesting pod/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Jul 28 22:45:33.586417 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:33.585998    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj" podUID="c474c0ba-3644-4872-9921-4bb7d49856a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Jul 28 22:45:34.585551 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:34.585507    2574 patch_prober.go:28] interesting pod/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Jul 28 22:45:34.585749 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:34.585573    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj" podUID="c474c0ba-3644-4872-9921-4bb7d49856a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Jul 28 22:45:35.590168 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:35.590132    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:35.590601 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:35.590482    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:35.591094 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:35.591077    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:45:44.587129 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:44.587094    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99"]
Jul 28 22:45:44.589463 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:44.589443    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99"
Jul 28 22:45:44.592058 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:44.592038    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-1661d9-serving-cert\""
Jul 28 22:45:44.624295 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:44.624265    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99"]
Jul 28 22:45:44.674785 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:44.674720    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8cff19b-7cca-4051-a2a6-2ae828c9868d-maas-api-tls\") pod \"maas-api-e2e-shared-b-1661d9-7d6999469-nvs99\" (UID: \"e8cff19b-7cca-4051-a2a6-2ae828c9868d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99"
Jul 28 22:45:44.675507 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:44.675465    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjpzk\" (UniqueName: \"kubernetes.io/projected/e8cff19b-7cca-4051-a2a6-2ae828c9868d-kube-api-access-bjpzk\") pod \"maas-api-e2e-shared-b-1661d9-7d6999469-nvs99\" (UID: \"e8cff19b-7cca-4051-a2a6-2ae828c9868d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99"
Jul 28 22:45:44.675659 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:44.675574    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8cff19b-7cca-4051-a2a6-2ae828c9868d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-1661d9-7d6999469-nvs99\" (UID: \"e8cff19b-7cca-4051-a2a6-2ae828c9868d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99"
Jul 28 22:45:44.776250 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:44.776195    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8cff19b-7cca-4051-a2a6-2ae828c9868d-maas-api-tls\") pod \"maas-api-e2e-shared-b-1661d9-7d6999469-nvs99\" (UID: \"e8cff19b-7cca-4051-a2a6-2ae828c9868d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99"
Jul 28 22:45:44.776418 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:44.776285    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjpzk\" (UniqueName: \"kubernetes.io/projected/e8cff19b-7cca-4051-a2a6-2ae828c9868d-kube-api-access-bjpzk\") pod \"maas-api-e2e-shared-b-1661d9-7d6999469-nvs99\" (UID: \"e8cff19b-7cca-4051-a2a6-2ae828c9868d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99"
Jul 28 22:45:44.776418 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:44.776331    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8cff19b-7cca-4051-a2a6-2ae828c9868d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-1661d9-7d6999469-nvs99\" (UID: \"e8cff19b-7cca-4051-a2a6-2ae828c9868d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99"
Jul 28 22:45:44.776418 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:45:44.776338    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-serving-cert: secret "maas-api-e2e-shared-b-1661d9-serving-cert" not found
Jul 28 22:45:44.776418 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:45:44.776411    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8cff19b-7cca-4051-a2a6-2ae828c9868d-maas-api-tls podName:e8cff19b-7cca-4051-a2a6-2ae828c9868d nodeName:}" failed. No retries permitted until 2026-07-28 22:45:45.276394369 +0000 UTC m=+2474.260536967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e8cff19b-7cca-4051-a2a6-2ae828c9868d-maas-api-tls") pod "maas-api-e2e-shared-b-1661d9-7d6999469-nvs99" (UID: "e8cff19b-7cca-4051-a2a6-2ae828c9868d") : secret "maas-api-e2e-shared-b-1661d9-serving-cert" not found
Jul 28 22:45:44.777028 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:44.776985    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8cff19b-7cca-4051-a2a6-2ae828c9868d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-1661d9-7d6999469-nvs99\" (UID: \"e8cff19b-7cca-4051-a2a6-2ae828c9868d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99"
Jul 28 22:45:44.785919 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:44.785891    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjpzk\" (UniqueName: \"kubernetes.io/projected/e8cff19b-7cca-4051-a2a6-2ae828c9868d-kube-api-access-bjpzk\") pod \"maas-api-e2e-shared-b-1661d9-7d6999469-nvs99\" (UID: \"e8cff19b-7cca-4051-a2a6-2ae828c9868d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99"
Jul 28 22:45:45.280813 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:45.280774    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8cff19b-7cca-4051-a2a6-2ae828c9868d-maas-api-tls\") pod \"maas-api-e2e-shared-b-1661d9-7d6999469-nvs99\" (UID: \"e8cff19b-7cca-4051-a2a6-2ae828c9868d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99"
Jul 28 22:45:45.283117 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:45.283092    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8cff19b-7cca-4051-a2a6-2ae828c9868d-maas-api-tls\") pod \"maas-api-e2e-shared-b-1661d9-7d6999469-nvs99\" (UID: \"e8cff19b-7cca-4051-a2a6-2ae828c9868d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99"
Jul 28 22:45:45.499431 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:45.499396    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99"
Jul 28 22:45:45.640171 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:45.640137    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99"]
Jul 28 22:45:45.642276 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:45:45.642219    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8cff19b_7cca_4051_a2a6_2ae828c9868d.slice/crio-070ef923a69c258aae724b4089f854dd547a0d275e705962d2524b219f2c2d03 WatchSource:0}: Error finding container 070ef923a69c258aae724b4089f854dd547a0d275e705962d2524b219f2c2d03: Status 404 returned error can't find the container with id 070ef923a69c258aae724b4089f854dd547a0d275e705962d2524b219f2c2d03
Jul 28 22:45:45.643742 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:45.643723    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 22:45:46.064675 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:46.064638    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99" event={"ID":"e8cff19b-7cca-4051-a2a6-2ae828c9868d","Type":"ContainerStarted","Data":"070ef923a69c258aae724b4089f854dd547a0d275e705962d2524b219f2c2d03"}
Jul 28 22:45:47.070188 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:47.070151    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99" event={"ID":"e8cff19b-7cca-4051-a2a6-2ae828c9868d","Type":"ContainerStarted","Data":"590eae72a35d53fd48343d11d87f26b2238aade90ec57ae8236bdeef63b21245"}
Jul 28 22:45:47.070594 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:47.070276    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99"
Jul 28 22:45:47.087829 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:47.087782    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99" podStartSLOduration=2.537291787 podStartE2EDuration="3.087768563s" totalImagesPullingTime="550.476776ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:45:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:45:47.086338139 +0000 UTC m=+2476.070480769" watchObservedRunningTime="2026-07-28 22:45:47.087768563 +0000 UTC m=+2476.071911181"
Jul 28 22:45:53.081000 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:45:53.080969    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99"
Jul 28 22:46:17.198043 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.198003    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"]
Jul 28 22:46:17.200483 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.200457    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:17.202714 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.202690    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ae4fa9\"/\"auth-test-model-ae4fa9-kserve-self-signed-certs\""
Jul 28 22:46:17.203214 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.203183    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ae4fa9\"/\"openshift-service-ca.crt\""
Jul 28 22:46:17.203344 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.203262    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ae4fa9\"/\"kube-root-ca.crt\""
Jul 28 22:46:17.203344 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.203333    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ae4fa9\"/\"default-dockercfg-ldnbm\""
Jul 28 22:46:17.214436 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.214412    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"]
Jul 28 22:46:17.263941 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.263905    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-dshm\") pod \"auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") " pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:17.264096 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.263958    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-tmp-dir\") pod \"auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") " pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:17.264096 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.263981    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2hbk\" (UniqueName: \"kubernetes.io/projected/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-kube-api-access-v2hbk\") pod \"auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") " pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:17.264096 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.264042    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-home\") pod \"auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") " pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:17.264222 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.264100    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-model-cache\") pod \"auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") " pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:17.264222 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.264117    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-tls-certs\") pod \"auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") " pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:17.364956 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.364918    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-model-cache\") pod \"auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") " pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:17.365121 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.364964    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-tls-certs\") pod \"auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") " pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:17.365121 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.365023    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-dshm\") pod \"auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") " pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:17.365121 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.365061    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-tmp-dir\") pod \"auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") " pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:17.365121 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.365088    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2hbk\" (UniqueName: \"kubernetes.io/projected/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-kube-api-access-v2hbk\") pod \"auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") " pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:17.365366 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.365134    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-home\") pod \"auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") " pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:17.365430 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.365374    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-model-cache\") pod \"auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") " pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:17.365570 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.365521    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-home\") pod \"auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") " pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:17.365660 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.365576    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-tmp-dir\") pod \"auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") " pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:17.367301 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.367281    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-dshm\") pod \"auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") " pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:17.367537 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.367514    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-tls-certs\") pod \"auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") " pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:17.374174 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.374146    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2hbk\" (UniqueName: \"kubernetes.io/projected/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-kube-api-access-v2hbk\") pod \"auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") " pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:17.511851 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.511757    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:17.641950 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:17.641922    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"]
Jul 28 22:46:17.644013 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:46:17.643980    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod614be4b3_9e98_4aeb_9cf7_50e18819bb3e.slice/crio-e9158fe3c237e89129a7d466357a59480b0e36b02eda7afc49793c63c085d3b6 WatchSource:0}: Error finding container e9158fe3c237e89129a7d466357a59480b0e36b02eda7afc49793c63c085d3b6: Status 404 returned error can't find the container with id e9158fe3c237e89129a7d466357a59480b0e36b02eda7afc49793c63c085d3b6
Jul 28 22:46:18.217543 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:18.217505    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn" event={"ID":"614be4b3-9e98-4aeb-9cf7-50e18819bb3e","Type":"ContainerStarted","Data":"e086a6d32226e8844e11b4705f3882a48fd4e5a1488577f7690c476fbcf84221"}
Jul 28 22:46:18.217543 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:18.217549    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn" event={"ID":"614be4b3-9e98-4aeb-9cf7-50e18819bb3e","Type":"ContainerStarted","Data":"e9158fe3c237e89129a7d466357a59480b0e36b02eda7afc49793c63c085d3b6"}
Jul 28 22:46:18.232931 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:18.232887    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn" podStartSLOduration=1.232871863 podStartE2EDuration="1.232871863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:46:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:46:18.232392884 +0000 UTC m=+2507.216535525" watchObservedRunningTime="2026-07-28 22:46:18.232871863 +0000 UTC m=+2507.217014483"
Jul 28 22:46:27.512633 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:27.512553    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:27.512633 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:27.512604    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:27.525020 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:27.524995    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:28.273624 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:28.273591    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:46:36.682148 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.682107    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"]
Jul 28 22:46:36.685920 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.685894    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:36.687906 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.687886    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1661d9\"/\"openshift-service-ca.crt\""
Jul 28 22:46:36.688503 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.688481    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1661d9\"/\"kube-root-ca.crt\""
Jul 28 22:46:36.688660 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.688520    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1661d9\"/\"auth-test-model-1661d9-kserve-self-signed-certs\""
Jul 28 22:46:36.688660 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.688587    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1661d9\"/\"default-dockercfg-fhr6m\""
Jul 28 22:46:36.695948 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.695925    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"]
Jul 28 22:46:36.848259 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.848191    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-model-cache\") pod \"auth-test-model-1661d9-kserve-69b988448c-kdk2q\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") " pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:36.848474 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.848266    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9brh\" (UniqueName: \"kubernetes.io/projected/936bf493-542c-45cc-a341-0bb4c11c995e-kube-api-access-w9brh\") pod \"auth-test-model-1661d9-kserve-69b988448c-kdk2q\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") " pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:36.848474 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.848329    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/936bf493-542c-45cc-a341-0bb4c11c995e-tls-certs\") pod \"auth-test-model-1661d9-kserve-69b988448c-kdk2q\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") " pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:36.848474 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.848370    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-home\") pod \"auth-test-model-1661d9-kserve-69b988448c-kdk2q\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") " pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:36.848474 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.848400    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-tmp-dir\") pod \"auth-test-model-1661d9-kserve-69b988448c-kdk2q\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") " pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:36.848647 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.848475    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-dshm\") pod \"auth-test-model-1661d9-kserve-69b988448c-kdk2q\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") " pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:36.949341 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.949306    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-home\") pod \"auth-test-model-1661d9-kserve-69b988448c-kdk2q\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") " pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:36.949546 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.949354    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-tmp-dir\") pod \"auth-test-model-1661d9-kserve-69b988448c-kdk2q\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") " pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:36.949546 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.949429    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-dshm\") pod \"auth-test-model-1661d9-kserve-69b988448c-kdk2q\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") " pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:36.949546 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.949482    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-model-cache\") pod \"auth-test-model-1661d9-kserve-69b988448c-kdk2q\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") " pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:36.949546 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.949511    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9brh\" (UniqueName: \"kubernetes.io/projected/936bf493-542c-45cc-a341-0bb4c11c995e-kube-api-access-w9brh\") pod \"auth-test-model-1661d9-kserve-69b988448c-kdk2q\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") " pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:36.949833 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.949572    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/936bf493-542c-45cc-a341-0bb4c11c995e-tls-certs\") pod \"auth-test-model-1661d9-kserve-69b988448c-kdk2q\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") " pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:36.949833 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.949710    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-home\") pod \"auth-test-model-1661d9-kserve-69b988448c-kdk2q\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") " pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:36.949833 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.949771    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-tmp-dir\") pod \"auth-test-model-1661d9-kserve-69b988448c-kdk2q\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") " pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:36.949988 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.949868    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-model-cache\") pod \"auth-test-model-1661d9-kserve-69b988448c-kdk2q\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") " pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:36.951877 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.951858    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-dshm\") pod \"auth-test-model-1661d9-kserve-69b988448c-kdk2q\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") " pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:36.952015 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.951997    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/936bf493-542c-45cc-a341-0bb4c11c995e-tls-certs\") pod \"auth-test-model-1661d9-kserve-69b988448c-kdk2q\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") " pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:36.958502 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.958478    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9brh\" (UniqueName: \"kubernetes.io/projected/936bf493-542c-45cc-a341-0bb4c11c995e-kube-api-access-w9brh\") pod \"auth-test-model-1661d9-kserve-69b988448c-kdk2q\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") " pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:36.998403 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:36.998361    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:37.138569 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:37.138544    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"]
Jul 28 22:46:37.140418 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:46:37.140378    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod936bf493_542c_45cc_a341_0bb4c11c995e.slice/crio-b37aee4c1656b9012a7abb8e63ea705508b47d360558a6cd39feebbec07a131e WatchSource:0}: Error finding container b37aee4c1656b9012a7abb8e63ea705508b47d360558a6cd39feebbec07a131e: Status 404 returned error can't find the container with id b37aee4c1656b9012a7abb8e63ea705508b47d360558a6cd39feebbec07a131e
Jul 28 22:46:37.307446 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:37.307358    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q" event={"ID":"936bf493-542c-45cc-a341-0bb4c11c995e","Type":"ContainerStarted","Data":"22f700ab0657927fee6896e0b95d8080effb0a0e160d5dc4e7907f5030f59326"}
Jul 28 22:46:37.307446 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:37.307402    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q" event={"ID":"936bf493-542c-45cc-a341-0bb4c11c995e","Type":"ContainerStarted","Data":"b37aee4c1656b9012a7abb8e63ea705508b47d360558a6cd39feebbec07a131e"}
Jul 28 22:46:37.327296 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:37.327244    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q" podStartSLOduration=1.327216213 podStartE2EDuration="1.327216213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:46:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:46:37.324974351 +0000 UTC m=+2526.309116984" watchObservedRunningTime="2026-07-28 22:46:37.327216213 +0000 UTC m=+2526.311358832"
Jul 28 22:46:46.998866 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:46.998827    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:46.999406 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:46.998881    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:47.011759 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:47.011727    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:46:47.368265 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:46:47.368169    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:51:06.390882 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:06.390806    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"]
Jul 28 22:51:07.345119 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:07.345081    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"]
Jul 28 22:51:21.409932 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.409890    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn" podUID="614be4b3-9e98-4aeb-9cf7-50e18819bb3e" containerName="main" containerID="cri-o://e086a6d32226e8844e11b4705f3882a48fd4e5a1488577f7690c476fbcf84221" gracePeriod=45
Jul 28 22:51:21.668944 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.668888    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:51:21.698424 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.698393    2574 generic.go:356] "Generic (PLEG): container finished" podID="614be4b3-9e98-4aeb-9cf7-50e18819bb3e" containerID="e086a6d32226e8844e11b4705f3882a48fd4e5a1488577f7690c476fbcf84221" exitCode=0
Jul 28 22:51:21.698585 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.698433    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"
Jul 28 22:51:21.698585 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.698477    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn" event={"ID":"614be4b3-9e98-4aeb-9cf7-50e18819bb3e","Type":"ContainerDied","Data":"e086a6d32226e8844e11b4705f3882a48fd4e5a1488577f7690c476fbcf84221"}
Jul 28 22:51:21.698585 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.698525    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn" event={"ID":"614be4b3-9e98-4aeb-9cf7-50e18819bb3e","Type":"ContainerDied","Data":"e9158fe3c237e89129a7d466357a59480b0e36b02eda7afc49793c63c085d3b6"}
Jul 28 22:51:21.698585 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.698547    2574 scope.go:117] "RemoveContainer" containerID="e086a6d32226e8844e11b4705f3882a48fd4e5a1488577f7690c476fbcf84221"
Jul 28 22:51:21.707822 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.707804    2574 scope.go:117] "RemoveContainer" containerID="e086a6d32226e8844e11b4705f3882a48fd4e5a1488577f7690c476fbcf84221"
Jul 28 22:51:21.708100 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:51:21.708080    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e086a6d32226e8844e11b4705f3882a48fd4e5a1488577f7690c476fbcf84221\": container with ID starting with e086a6d32226e8844e11b4705f3882a48fd4e5a1488577f7690c476fbcf84221 not found: ID does not exist" containerID="e086a6d32226e8844e11b4705f3882a48fd4e5a1488577f7690c476fbcf84221"
Jul 28 22:51:21.708179 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.708111    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e086a6d32226e8844e11b4705f3882a48fd4e5a1488577f7690c476fbcf84221"} err="failed to get container status \"e086a6d32226e8844e11b4705f3882a48fd4e5a1488577f7690c476fbcf84221\": rpc error: code = NotFound desc = could not find container \"e086a6d32226e8844e11b4705f3882a48fd4e5a1488577f7690c476fbcf84221\": container with ID starting with e086a6d32226e8844e11b4705f3882a48fd4e5a1488577f7690c476fbcf84221 not found: ID does not exist"
Jul 28 22:51:21.795135 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.795101    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-home\") pod \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") "
Jul 28 22:51:21.795342 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.795185    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-tls-certs\") pod \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") "
Jul 28 22:51:21.795342 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.795267    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-dshm\") pod \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") "
Jul 28 22:51:21.795342 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.795292    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2hbk\" (UniqueName: \"kubernetes.io/projected/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-kube-api-access-v2hbk\") pod \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") "
Jul 28 22:51:21.795342 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.795314    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-model-cache\") pod \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") "
Jul 28 22:51:21.795342 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.795333    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-tmp-dir\") pod \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\" (UID: \"614be4b3-9e98-4aeb-9cf7-50e18819bb3e\") "
Jul 28 22:51:21.795628 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.795426    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-home" (OuterVolumeSpecName: "home") pod "614be4b3-9e98-4aeb-9cf7-50e18819bb3e" (UID: "614be4b3-9e98-4aeb-9cf7-50e18819bb3e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:51:21.795686 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.795633    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-model-cache" (OuterVolumeSpecName: "model-cache") pod "614be4b3-9e98-4aeb-9cf7-50e18819bb3e" (UID: "614be4b3-9e98-4aeb-9cf7-50e18819bb3e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:51:21.795686 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.795636    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-home\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:51:21.795686 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.795643    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "614be4b3-9e98-4aeb-9cf7-50e18819bb3e" (UID: "614be4b3-9e98-4aeb-9cf7-50e18819bb3e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:51:21.797662 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.797635    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "614be4b3-9e98-4aeb-9cf7-50e18819bb3e" (UID: "614be4b3-9e98-4aeb-9cf7-50e18819bb3e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:51:21.797662 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.797649    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-dshm" (OuterVolumeSpecName: "dshm") pod "614be4b3-9e98-4aeb-9cf7-50e18819bb3e" (UID: "614be4b3-9e98-4aeb-9cf7-50e18819bb3e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:51:21.797810 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.797687    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-kube-api-access-v2hbk" (OuterVolumeSpecName: "kube-api-access-v2hbk") pod "614be4b3-9e98-4aeb-9cf7-50e18819bb3e" (UID: "614be4b3-9e98-4aeb-9cf7-50e18819bb3e"). InnerVolumeSpecName "kube-api-access-v2hbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:51:21.896552 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.896521    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-tls-certs\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:51:21.896552 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.896545    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-dshm\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:51:21.896552 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.896555    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2hbk\" (UniqueName: \"kubernetes.io/projected/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-kube-api-access-v2hbk\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:51:21.896764 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.896564    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-model-cache\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:51:21.896764 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:21.896573    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/614be4b3-9e98-4aeb-9cf7-50e18819bb3e-tmp-dir\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:51:22.018651 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.018623    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"]
Jul 28 22:51:22.027420 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.024115    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ae4fa9/auth-test-model-ae4fa9-kserve-66578cb7ff-6xpqn"]
Jul 28 22:51:22.364560 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.364471    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q" podUID="936bf493-542c-45cc-a341-0bb4c11c995e" containerName="main" containerID="cri-o://22f700ab0657927fee6896e0b95d8080effb0a0e160d5dc4e7907f5030f59326" gracePeriod=45
Jul 28 22:51:22.621918 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.621854    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:51:22.702004 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.701978    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9brh\" (UniqueName: \"kubernetes.io/projected/936bf493-542c-45cc-a341-0bb4c11c995e-kube-api-access-w9brh\") pod \"936bf493-542c-45cc-a341-0bb4c11c995e\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") "
Jul 28 22:51:22.702004 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.702010    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-dshm\") pod \"936bf493-542c-45cc-a341-0bb4c11c995e\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") "
Jul 28 22:51:22.702289 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.702037    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-home\") pod \"936bf493-542c-45cc-a341-0bb4c11c995e\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") "
Jul 28 22:51:22.702289 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.702059    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-tmp-dir\") pod \"936bf493-542c-45cc-a341-0bb4c11c995e\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") "
Jul 28 22:51:22.702289 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.702124    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-model-cache\") pod \"936bf493-542c-45cc-a341-0bb4c11c995e\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") "
Jul 28 22:51:22.702289 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.702186    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/936bf493-542c-45cc-a341-0bb4c11c995e-tls-certs\") pod \"936bf493-542c-45cc-a341-0bb4c11c995e\" (UID: \"936bf493-542c-45cc-a341-0bb4c11c995e\") "
Jul 28 22:51:22.702501 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.702281    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-home" (OuterVolumeSpecName: "home") pod "936bf493-542c-45cc-a341-0bb4c11c995e" (UID: "936bf493-542c-45cc-a341-0bb4c11c995e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:51:22.702501 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.702361    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "936bf493-542c-45cc-a341-0bb4c11c995e" (UID: "936bf493-542c-45cc-a341-0bb4c11c995e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:51:22.702602 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.702492    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-model-cache" (OuterVolumeSpecName: "model-cache") pod "936bf493-542c-45cc-a341-0bb4c11c995e" (UID: "936bf493-542c-45cc-a341-0bb4c11c995e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:51:22.702602 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.702558    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-home\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:51:22.702677 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.702611    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-tmp-dir\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:51:22.704475 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.704446    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/936bf493-542c-45cc-a341-0bb4c11c995e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "936bf493-542c-45cc-a341-0bb4c11c995e" (UID: "936bf493-542c-45cc-a341-0bb4c11c995e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:51:22.704608 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.704522    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-dshm" (OuterVolumeSpecName: "dshm") pod "936bf493-542c-45cc-a341-0bb4c11c995e" (UID: "936bf493-542c-45cc-a341-0bb4c11c995e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:51:22.704692 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.704668    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/936bf493-542c-45cc-a341-0bb4c11c995e-kube-api-access-w9brh" (OuterVolumeSpecName: "kube-api-access-w9brh") pod "936bf493-542c-45cc-a341-0bb4c11c995e" (UID: "936bf493-542c-45cc-a341-0bb4c11c995e"). InnerVolumeSpecName "kube-api-access-w9brh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:51:22.706870 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.706842    2574 generic.go:356] "Generic (PLEG): container finished" podID="936bf493-542c-45cc-a341-0bb4c11c995e" containerID="22f700ab0657927fee6896e0b95d8080effb0a0e160d5dc4e7907f5030f59326" exitCode=0
Jul 28 22:51:22.706996 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.706884    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"
Jul 28 22:51:22.706996 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.706896    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q" event={"ID":"936bf493-542c-45cc-a341-0bb4c11c995e","Type":"ContainerDied","Data":"22f700ab0657927fee6896e0b95d8080effb0a0e160d5dc4e7907f5030f59326"}
Jul 28 22:51:22.706996 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.706935    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q" event={"ID":"936bf493-542c-45cc-a341-0bb4c11c995e","Type":"ContainerDied","Data":"b37aee4c1656b9012a7abb8e63ea705508b47d360558a6cd39feebbec07a131e"}
Jul 28 22:51:22.706996 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.706958    2574 scope.go:117] "RemoveContainer" containerID="22f700ab0657927fee6896e0b95d8080effb0a0e160d5dc4e7907f5030f59326"
Jul 28 22:51:22.722487 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.722466    2574 scope.go:117] "RemoveContainer" containerID="22f700ab0657927fee6896e0b95d8080effb0a0e160d5dc4e7907f5030f59326"
Jul 28 22:51:22.722741 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:51:22.722723    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22f700ab0657927fee6896e0b95d8080effb0a0e160d5dc4e7907f5030f59326\": container with ID starting with 22f700ab0657927fee6896e0b95d8080effb0a0e160d5dc4e7907f5030f59326 not found: ID does not exist" containerID="22f700ab0657927fee6896e0b95d8080effb0a0e160d5dc4e7907f5030f59326"
Jul 28 22:51:22.722787 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.722749    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22f700ab0657927fee6896e0b95d8080effb0a0e160d5dc4e7907f5030f59326"} err="failed to get container status \"22f700ab0657927fee6896e0b95d8080effb0a0e160d5dc4e7907f5030f59326\": rpc error: code = NotFound desc = could not find container \"22f700ab0657927fee6896e0b95d8080effb0a0e160d5dc4e7907f5030f59326\": container with ID starting with 22f700ab0657927fee6896e0b95d8080effb0a0e160d5dc4e7907f5030f59326 not found: ID does not exist"
Jul 28 22:51:22.733823 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.733791    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"]
Jul 28 22:51:22.736962 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.736925    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1661d9/auth-test-model-1661d9-kserve-69b988448c-kdk2q"]
Jul 28 22:51:22.803978 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.803951    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/936bf493-542c-45cc-a341-0bb4c11c995e-tls-certs\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:51:22.803978 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.803976    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9brh\" (UniqueName: \"kubernetes.io/projected/936bf493-542c-45cc-a341-0bb4c11c995e-kube-api-access-w9brh\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:51:22.804169 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.803986    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-dshm\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:51:22.804169 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:22.803995    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/936bf493-542c-45cc-a341-0bb4c11c995e-model-cache\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:51:23.493456 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:23.493424    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="614be4b3-9e98-4aeb-9cf7-50e18819bb3e" path="/var/lib/kubelet/pods/614be4b3-9e98-4aeb-9cf7-50e18819bb3e/volumes"
Jul 28 22:51:23.493821 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:23.493808    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="936bf493-542c-45cc-a341-0bb4c11c995e" path="/var/lib/kubelet/pods/936bf493-542c-45cc-a341-0bb4c11c995e/volumes"
Jul 28 22:51:30.003823 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.003790    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"]
Jul 28 22:51:30.004214 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.004140    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="936bf493-542c-45cc-a341-0bb4c11c995e" containerName="main"
Jul 28 22:51:30.004214 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.004152    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="936bf493-542c-45cc-a341-0bb4c11c995e" containerName="main"
Jul 28 22:51:30.004214 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.004166    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="614be4b3-9e98-4aeb-9cf7-50e18819bb3e" containerName="main"
Jul 28 22:51:30.004214 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.004171    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="614be4b3-9e98-4aeb-9cf7-50e18819bb3e" containerName="main"
Jul 28 22:51:30.004397 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.004245    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="936bf493-542c-45cc-a341-0bb4c11c995e" containerName="main"
Jul 28 22:51:30.004397 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.004254    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="614be4b3-9e98-4aeb-9cf7-50e18819bb3e" containerName="main"
Jul 28 22:51:30.007394 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.007376    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:30.009702 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.009677    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ae4fa9\"/\"sub-test-model-ae4fa9-kserve-self-signed-certs\""
Jul 28 22:51:30.009832 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.009702    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ae4fa9\"/\"default-dockercfg-ldnbm\""
Jul 28 22:51:30.009832 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.009702    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ae4fa9\"/\"kube-root-ca.crt\""
Jul 28 22:51:30.009832 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.009728    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ae4fa9\"/\"openshift-service-ca.crt\""
Jul 28 22:51:30.016447 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.016422    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"]
Jul 28 22:51:30.169617 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.169588    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-dshm\") pod \"sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") " pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:30.169791 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.169649    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-tmp-dir\") pod \"sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") " pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:30.169791 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.169688    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fdrf\" (UniqueName: \"kubernetes.io/projected/2047c26e-b66e-4626-9399-1c38d82669b8-kube-api-access-7fdrf\") pod \"sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") " pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:30.169791 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.169747    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-home\") pod \"sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") " pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:30.169791 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.169765    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-model-cache\") pod \"sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") " pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:30.169942 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.169793    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2047c26e-b66e-4626-9399-1c38d82669b8-tls-certs\") pod \"sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") " pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:30.270831 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.270749    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-tmp-dir\") pod \"sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") " pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:30.270831 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.270782    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fdrf\" (UniqueName: \"kubernetes.io/projected/2047c26e-b66e-4626-9399-1c38d82669b8-kube-api-access-7fdrf\") pod \"sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") " pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:30.270831 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.270822    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-home\") pod \"sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") " pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:30.271106 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.270836    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-model-cache\") pod \"sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") " pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:30.271106 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.270871    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2047c26e-b66e-4626-9399-1c38d82669b8-tls-certs\") pod \"sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") " pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:30.271106 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.270898    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-dshm\") pod \"sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") " pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:30.271285 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.271199    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-home\") pod \"sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") " pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:30.271346 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.271279    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-model-cache\") pod \"sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") " pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:30.271410 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.271385    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-tmp-dir\") pod \"sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") " pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:30.273008 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.272986    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-dshm\") pod \"sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") " pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:30.273296 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.273278    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2047c26e-b66e-4626-9399-1c38d82669b8-tls-certs\") pod \"sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") " pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:30.277669 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.277643    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fdrf\" (UniqueName: \"kubernetes.io/projected/2047c26e-b66e-4626-9399-1c38d82669b8-kube-api-access-7fdrf\") pod \"sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") " pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:30.317472 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.317441    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:30.452015 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.451977    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"]
Jul 28 22:51:30.455184 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:51:30.455157    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2047c26e_b66e_4626_9399_1c38d82669b8.slice/crio-070a2ebf30f6eb7cbbd9dfee62edd3456437faca23de94edc54f3088e1ab0623 WatchSource:0}: Error finding container 070a2ebf30f6eb7cbbd9dfee62edd3456437faca23de94edc54f3088e1ab0623: Status 404 returned error can't find the container with id 070a2ebf30f6eb7cbbd9dfee62edd3456437faca23de94edc54f3088e1ab0623
Jul 28 22:51:30.749308 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.749270    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw" event={"ID":"2047c26e-b66e-4626-9399-1c38d82669b8","Type":"ContainerStarted","Data":"3c94a2b6ce8d14e5f6907616631a1c2b71f7e3c8215346d1f0e92cadd43ec721"}
Jul 28 22:51:30.749481 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.749321    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw" event={"ID":"2047c26e-b66e-4626-9399-1c38d82669b8","Type":"ContainerStarted","Data":"070a2ebf30f6eb7cbbd9dfee62edd3456437faca23de94edc54f3088e1ab0623"}
Jul 28 22:51:30.768309 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:30.768262    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw" podStartSLOduration=1.768245888 podStartE2EDuration="1.768245888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:51:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:51:30.766486759 +0000 UTC m=+2819.750629392" watchObservedRunningTime="2026-07-28 22:51:30.768245888 +0000 UTC m=+2819.752388563"
Jul 28 22:51:40.318607 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:40.318563    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:40.319196 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:40.318628    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:40.331095 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:40.331068    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:40.804800 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:40.804770    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:51:46.502346 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.502301    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"]
Jul 28 22:51:46.505715 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.505694    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:46.507877 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.507853    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1661d9\"/\"openshift-service-ca.crt\""
Jul 28 22:51:46.507976 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.507904    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1661d9\"/\"kube-root-ca.crt\""
Jul 28 22:51:46.507976 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.507952    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1661d9\"/\"sub-test-model-1661d9-kserve-self-signed-certs\""
Jul 28 22:51:46.508303 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.508287    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1661d9\"/\"default-dockercfg-fhr6m\""
Jul 28 22:51:46.515922 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.515900    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"]
Jul 28 22:51:46.517625 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.517599    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7htxx\" (UniqueName: \"kubernetes.io/projected/b090e421-6041-476a-b320-04bc01ed3967-kube-api-access-7htxx\") pod \"sub-test-model-1661d9-kserve-7446f8f689-5llmt\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") " pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:46.517720 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.517641    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-model-cache\") pod \"sub-test-model-1661d9-kserve-7446f8f689-5llmt\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") " pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:46.517720 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.517705    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-dshm\") pod \"sub-test-model-1661d9-kserve-7446f8f689-5llmt\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") " pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:46.517815 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.517760    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-home\") pod \"sub-test-model-1661d9-kserve-7446f8f689-5llmt\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") " pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:46.517858 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.517825    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b090e421-6041-476a-b320-04bc01ed3967-tls-certs\") pod \"sub-test-model-1661d9-kserve-7446f8f689-5llmt\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") " pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:46.517932 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.517857    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-tmp-dir\") pod \"sub-test-model-1661d9-kserve-7446f8f689-5llmt\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") " pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:46.618504 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.618451    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b090e421-6041-476a-b320-04bc01ed3967-tls-certs\") pod \"sub-test-model-1661d9-kserve-7446f8f689-5llmt\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") " pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:46.618504 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.618506    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-tmp-dir\") pod \"sub-test-model-1661d9-kserve-7446f8f689-5llmt\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") " pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:46.618782 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.618564    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7htxx\" (UniqueName: \"kubernetes.io/projected/b090e421-6041-476a-b320-04bc01ed3967-kube-api-access-7htxx\") pod \"sub-test-model-1661d9-kserve-7446f8f689-5llmt\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") " pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:46.618782 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.618582    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-model-cache\") pod \"sub-test-model-1661d9-kserve-7446f8f689-5llmt\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") " pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:46.618782 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.618598    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-dshm\") pod \"sub-test-model-1661d9-kserve-7446f8f689-5llmt\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") " pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:46.618782 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.618621    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-home\") pod \"sub-test-model-1661d9-kserve-7446f8f689-5llmt\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") " pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:46.619001 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.618982    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-tmp-dir\") pod \"sub-test-model-1661d9-kserve-7446f8f689-5llmt\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") " pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:46.619061 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.619043    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-home\") pod \"sub-test-model-1661d9-kserve-7446f8f689-5llmt\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") " pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:46.619114 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.619059    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-model-cache\") pod \"sub-test-model-1661d9-kserve-7446f8f689-5llmt\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") " pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:46.620912 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.620886    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-dshm\") pod \"sub-test-model-1661d9-kserve-7446f8f689-5llmt\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") " pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:46.621107 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.621090    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b090e421-6041-476a-b320-04bc01ed3967-tls-certs\") pod \"sub-test-model-1661d9-kserve-7446f8f689-5llmt\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") " pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:46.626326 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.626308    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7htxx\" (UniqueName: \"kubernetes.io/projected/b090e421-6041-476a-b320-04bc01ed3967-kube-api-access-7htxx\") pod \"sub-test-model-1661d9-kserve-7446f8f689-5llmt\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") " pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:46.815961 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.815877    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:46.949000 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:46.948971    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"]
Jul 28 22:51:46.951066 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:51:46.951040    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb090e421_6041_476a_b320_04bc01ed3967.slice/crio-4825650efff510a93eb07d3f85505e31117c3cc206e1c268bf32bfcef37d3de9 WatchSource:0}: Error finding container 4825650efff510a93eb07d3f85505e31117c3cc206e1c268bf32bfcef37d3de9: Status 404 returned error can't find the container with id 4825650efff510a93eb07d3f85505e31117c3cc206e1c268bf32bfcef37d3de9
Jul 28 22:51:47.831286 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:47.831249    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt" event={"ID":"b090e421-6041-476a-b320-04bc01ed3967","Type":"ContainerStarted","Data":"29111efb6a9652fc83923d263b237ab6d7c4173e413ff72545f722b94df783ca"}
Jul 28 22:51:47.831286 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:47.831291    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt" event={"ID":"b090e421-6041-476a-b320-04bc01ed3967","Type":"ContainerStarted","Data":"4825650efff510a93eb07d3f85505e31117c3cc206e1c268bf32bfcef37d3de9"}
Jul 28 22:51:47.849371 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:47.849301    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt" podStartSLOduration=1.849282528 podStartE2EDuration="1.849282528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:51:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:51:47.84869048 +0000 UTC m=+2836.832833113" watchObservedRunningTime="2026-07-28 22:51:47.849282528 +0000 UTC m=+2836.833425157"
Jul 28 22:51:56.816906 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:56.816870    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:56.817521 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:56.816919    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:56.829564 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:56.829537    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:51:56.887710 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:51:56.887685    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:52:49.208047 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:52:49.208016    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"]
Jul 28 22:52:50.122313 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:52:50.122280    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"]
Jul 28 22:53:00.702899 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:00.702870    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"]
Jul 28 22:53:00.706466 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:00.706444    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:00.708436 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:00.708417    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ae4fa9\"/\"rate-test-model-ae4fa9-kserve-self-signed-certs\""
Jul 28 22:53:00.718807 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:00.718786    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"]
Jul 28 22:53:00.781166 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:00.781137    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-tmp-dir\") pod \"rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") " pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:00.781353 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:00.781206    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mh6k\" (UniqueName: \"kubernetes.io/projected/fe20fe90-fa5e-483a-b78c-e445c2c098e2-kube-api-access-6mh6k\") pod \"rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") " pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:00.781353 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:00.781318    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe20fe90-fa5e-483a-b78c-e445c2c098e2-tls-certs\") pod \"rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") " pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:00.781443 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:00.781369    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-model-cache\") pod \"rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") " pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:00.781443 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:00.781388    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-dshm\") pod \"rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") " pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:00.781508 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:00.781464    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-home\") pod \"rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") " pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:00.882227 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:00.882194    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mh6k\" (UniqueName: \"kubernetes.io/projected/fe20fe90-fa5e-483a-b78c-e445c2c098e2-kube-api-access-6mh6k\") pod \"rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") " pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:00.882426 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:00.882270    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe20fe90-fa5e-483a-b78c-e445c2c098e2-tls-certs\") pod \"rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") " pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:00.882426 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:00.882300    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-model-cache\") pod \"rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") " pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:00.882426 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:00.882317    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-dshm\") pod \"rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") " pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:00.882426 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:00.882383    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-home\") pod \"rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") " pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:00.882426 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:00.882421    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-tmp-dir\") pod \"rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") " pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:00.882758 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:00.882727    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-model-cache\") pod \"rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") " pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:00.882821 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:00.882786    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-home\") pod \"rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") " pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:00.882862 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:00.882838    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-tmp-dir\") pod \"rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") " pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:00.884601 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:00.884580    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-dshm\") pod \"rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") " pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:00.884757 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:00.884738    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe20fe90-fa5e-483a-b78c-e445c2c098e2-tls-certs\") pod \"rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") " pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:00.890075 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:00.890053    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mh6k\" (UniqueName: \"kubernetes.io/projected/fe20fe90-fa5e-483a-b78c-e445c2c098e2-kube-api-access-6mh6k\") pod \"rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") " pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:01.015625 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:01.015532    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:01.149851 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:01.149641    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"]
Jul 28 22:53:01.152803 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:53:01.152771    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe20fe90_fa5e_483a_b78c_e445c2c098e2.slice/crio-0eef2b30a06c76e36ee0f8b00dc4c45a1b502cf6a5ad1066aacdda77ba299425 WatchSource:0}: Error finding container 0eef2b30a06c76e36ee0f8b00dc4c45a1b502cf6a5ad1066aacdda77ba299425: Status 404 returned error can't find the container with id 0eef2b30a06c76e36ee0f8b00dc4c45a1b502cf6a5ad1066aacdda77ba299425
Jul 28 22:53:01.186174 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:01.186144    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4" event={"ID":"fe20fe90-fa5e-483a-b78c-e445c2c098e2","Type":"ContainerStarted","Data":"0eef2b30a06c76e36ee0f8b00dc4c45a1b502cf6a5ad1066aacdda77ba299425"}
Jul 28 22:53:02.192402 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:02.192362    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4" event={"ID":"fe20fe90-fa5e-483a-b78c-e445c2c098e2","Type":"ContainerStarted","Data":"4f5bee513250f1281bca39df776267cdede491b47576c889312d6b3fdbfcdee2"}
Jul 28 22:53:02.207931 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:02.207881    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4" podStartSLOduration=2.207865007 podStartE2EDuration="2.207865007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:53:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:53:02.206571928 +0000 UTC m=+2911.190714546" watchObservedRunningTime="2026-07-28 22:53:02.207865007 +0000 UTC m=+2911.192007625"
Jul 28 22:53:04.226485 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:04.226447    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw" podUID="2047c26e-b66e-4626-9399-1c38d82669b8" containerName="main" containerID="cri-o://3c94a2b6ce8d14e5f6907616631a1c2b71f7e3c8215346d1f0e92cadd43ec721" gracePeriod=45
Jul 28 22:53:04.486844 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:04.486783    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:53:04.616494 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:04.616452    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-model-cache\") pod \"2047c26e-b66e-4626-9399-1c38d82669b8\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") "
Jul 28 22:53:04.616699 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:04.616511    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-tmp-dir\") pod \"2047c26e-b66e-4626-9399-1c38d82669b8\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") "
Jul 28 22:53:04.616699 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:04.616566    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-dshm\") pod \"2047c26e-b66e-4626-9399-1c38d82669b8\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") "
Jul 28 22:53:04.616699 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:04.616585    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fdrf\" (UniqueName: \"kubernetes.io/projected/2047c26e-b66e-4626-9399-1c38d82669b8-kube-api-access-7fdrf\") pod \"2047c26e-b66e-4626-9399-1c38d82669b8\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") "
Jul 28 22:53:04.616699 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:04.616642    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2047c26e-b66e-4626-9399-1c38d82669b8-tls-certs\") pod \"2047c26e-b66e-4626-9399-1c38d82669b8\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") "
Jul 28 22:53:04.616920 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:04.616719    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-home\") pod \"2047c26e-b66e-4626-9399-1c38d82669b8\" (UID: \"2047c26e-b66e-4626-9399-1c38d82669b8\") "
Jul 28 22:53:04.616920 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:04.616743    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-model-cache" (OuterVolumeSpecName: "model-cache") pod "2047c26e-b66e-4626-9399-1c38d82669b8" (UID: "2047c26e-b66e-4626-9399-1c38d82669b8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:53:04.616920 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:04.616788    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2047c26e-b66e-4626-9399-1c38d82669b8" (UID: "2047c26e-b66e-4626-9399-1c38d82669b8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:53:04.617042 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:04.616943    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-model-cache\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:53:04.617042 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:04.616951    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-home" (OuterVolumeSpecName: "home") pod "2047c26e-b66e-4626-9399-1c38d82669b8" (UID: "2047c26e-b66e-4626-9399-1c38d82669b8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:53:04.617042 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:04.616957    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-tmp-dir\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:53:04.618735 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:04.618714    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2047c26e-b66e-4626-9399-1c38d82669b8-kube-api-access-7fdrf" (OuterVolumeSpecName: "kube-api-access-7fdrf") pod "2047c26e-b66e-4626-9399-1c38d82669b8" (UID: "2047c26e-b66e-4626-9399-1c38d82669b8"). InnerVolumeSpecName "kube-api-access-7fdrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:53:04.618818 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:04.618733    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2047c26e-b66e-4626-9399-1c38d82669b8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2047c26e-b66e-4626-9399-1c38d82669b8" (UID: "2047c26e-b66e-4626-9399-1c38d82669b8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:53:04.618818 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:04.618786    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-dshm" (OuterVolumeSpecName: "dshm") pod "2047c26e-b66e-4626-9399-1c38d82669b8" (UID: "2047c26e-b66e-4626-9399-1c38d82669b8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:53:04.718245 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:04.718200    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-dshm\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:53:04.718245 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:04.718247    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fdrf\" (UniqueName: \"kubernetes.io/projected/2047c26e-b66e-4626-9399-1c38d82669b8-kube-api-access-7fdrf\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:53:04.718451 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:04.718261    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2047c26e-b66e-4626-9399-1c38d82669b8-tls-certs\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:53:04.718451 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:04.718274    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2047c26e-b66e-4626-9399-1c38d82669b8-home\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:53:05.141852 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.141805    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt" podUID="b090e421-6041-476a-b320-04bc01ed3967" containerName="main" containerID="cri-o://29111efb6a9652fc83923d263b237ab6d7c4173e413ff72545f722b94df783ca" gracePeriod=45
Jul 28 22:53:05.207586 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.207552    2574 generic.go:356] "Generic (PLEG): container finished" podID="2047c26e-b66e-4626-9399-1c38d82669b8" containerID="3c94a2b6ce8d14e5f6907616631a1c2b71f7e3c8215346d1f0e92cadd43ec721" exitCode=0
Jul 28 22:53:05.207744 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.207599    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"
Jul 28 22:53:05.207744 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.207613    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw" event={"ID":"2047c26e-b66e-4626-9399-1c38d82669b8","Type":"ContainerDied","Data":"3c94a2b6ce8d14e5f6907616631a1c2b71f7e3c8215346d1f0e92cadd43ec721"}
Jul 28 22:53:05.207744 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.207648    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw" event={"ID":"2047c26e-b66e-4626-9399-1c38d82669b8","Type":"ContainerDied","Data":"070a2ebf30f6eb7cbbd9dfee62edd3456437faca23de94edc54f3088e1ab0623"}
Jul 28 22:53:05.207744 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.207669    2574 scope.go:117] "RemoveContainer" containerID="3c94a2b6ce8d14e5f6907616631a1c2b71f7e3c8215346d1f0e92cadd43ec721"
Jul 28 22:53:05.226463 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.226438    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"]
Jul 28 22:53:05.228197 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.228178    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ae4fa9/sub-test-model-ae4fa9-kserve-6fbc569bb7-4klhw"]
Jul 28 22:53:05.252678 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.252565    2574 scope.go:117] "RemoveContainer" containerID="3c94a2b6ce8d14e5f6907616631a1c2b71f7e3c8215346d1f0e92cadd43ec721"
Jul 28 22:53:05.252952 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:53:05.252931    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c94a2b6ce8d14e5f6907616631a1c2b71f7e3c8215346d1f0e92cadd43ec721\": container with ID starting with 3c94a2b6ce8d14e5f6907616631a1c2b71f7e3c8215346d1f0e92cadd43ec721 not found: ID does not exist" containerID="3c94a2b6ce8d14e5f6907616631a1c2b71f7e3c8215346d1f0e92cadd43ec721"
Jul 28 22:53:05.252991 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.252964    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c94a2b6ce8d14e5f6907616631a1c2b71f7e3c8215346d1f0e92cadd43ec721"} err="failed to get container status \"3c94a2b6ce8d14e5f6907616631a1c2b71f7e3c8215346d1f0e92cadd43ec721\": rpc error: code = NotFound desc = could not find container \"3c94a2b6ce8d14e5f6907616631a1c2b71f7e3c8215346d1f0e92cadd43ec721\": container with ID starting with 3c94a2b6ce8d14e5f6907616631a1c2b71f7e3c8215346d1f0e92cadd43ec721 not found: ID does not exist"
Jul 28 22:53:05.393796 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.393735    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:53:05.494750 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.494719    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2047c26e-b66e-4626-9399-1c38d82669b8" path="/var/lib/kubelet/pods/2047c26e-b66e-4626-9399-1c38d82669b8/volumes"
Jul 28 22:53:05.527292 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.527226    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7htxx\" (UniqueName: \"kubernetes.io/projected/b090e421-6041-476a-b320-04bc01ed3967-kube-api-access-7htxx\") pod \"b090e421-6041-476a-b320-04bc01ed3967\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") "
Jul 28 22:53:05.527439 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.527327    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-model-cache\") pod \"b090e421-6041-476a-b320-04bc01ed3967\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") "
Jul 28 22:53:05.527439 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.527348    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b090e421-6041-476a-b320-04bc01ed3967-tls-certs\") pod \"b090e421-6041-476a-b320-04bc01ed3967\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") "
Jul 28 22:53:05.527439 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.527372    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-home\") pod \"b090e421-6041-476a-b320-04bc01ed3967\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") "
Jul 28 22:53:05.527439 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.527416    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-tmp-dir\") pod \"b090e421-6041-476a-b320-04bc01ed3967\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") "
Jul 28 22:53:05.527628 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.527533    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-dshm\") pod \"b090e421-6041-476a-b320-04bc01ed3967\" (UID: \"b090e421-6041-476a-b320-04bc01ed3967\") "
Jul 28 22:53:05.527628 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.527596    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-model-cache" (OuterVolumeSpecName: "model-cache") pod "b090e421-6041-476a-b320-04bc01ed3967" (UID: "b090e421-6041-476a-b320-04bc01ed3967"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:53:05.527722 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.527670    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b090e421-6041-476a-b320-04bc01ed3967" (UID: "b090e421-6041-476a-b320-04bc01ed3967"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:53:05.527722 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.527683    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-home" (OuterVolumeSpecName: "home") pod "b090e421-6041-476a-b320-04bc01ed3967" (UID: "b090e421-6041-476a-b320-04bc01ed3967"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:53:05.527845 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.527832    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-model-cache\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:53:05.527900 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.527848    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-home\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:53:05.527900 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.527856    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-tmp-dir\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:53:05.529452 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.529423    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b090e421-6041-476a-b320-04bc01ed3967-kube-api-access-7htxx" (OuterVolumeSpecName: "kube-api-access-7htxx") pod "b090e421-6041-476a-b320-04bc01ed3967" (UID: "b090e421-6041-476a-b320-04bc01ed3967"). InnerVolumeSpecName "kube-api-access-7htxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:53:05.529452 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.529427    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b090e421-6041-476a-b320-04bc01ed3967-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b090e421-6041-476a-b320-04bc01ed3967" (UID: "b090e421-6041-476a-b320-04bc01ed3967"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:53:05.529632 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.529559    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-dshm" (OuterVolumeSpecName: "dshm") pod "b090e421-6041-476a-b320-04bc01ed3967" (UID: "b090e421-6041-476a-b320-04bc01ed3967"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:53:05.628492 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.628460    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b090e421-6041-476a-b320-04bc01ed3967-tls-certs\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:53:05.628492 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.628489    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b090e421-6041-476a-b320-04bc01ed3967-dshm\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:53:05.628492 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:05.628500    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7htxx\" (UniqueName: \"kubernetes.io/projected/b090e421-6041-476a-b320-04bc01ed3967-kube-api-access-7htxx\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:53:06.214080 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:06.214042    2574 generic.go:356] "Generic (PLEG): container finished" podID="b090e421-6041-476a-b320-04bc01ed3967" containerID="29111efb6a9652fc83923d263b237ab6d7c4173e413ff72545f722b94df783ca" exitCode=0
Jul 28 22:53:06.214080 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:06.214073    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt" event={"ID":"b090e421-6041-476a-b320-04bc01ed3967","Type":"ContainerDied","Data":"29111efb6a9652fc83923d263b237ab6d7c4173e413ff72545f722b94df783ca"}
Jul 28 22:53:06.214374 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:06.214116    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt" event={"ID":"b090e421-6041-476a-b320-04bc01ed3967","Type":"ContainerDied","Data":"4825650efff510a93eb07d3f85505e31117c3cc206e1c268bf32bfcef37d3de9"}
Jul 28 22:53:06.214374 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:06.214116    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"
Jul 28 22:53:06.214374 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:06.214137    2574 scope.go:117] "RemoveContainer" containerID="29111efb6a9652fc83923d263b237ab6d7c4173e413ff72545f722b94df783ca"
Jul 28 22:53:06.223660 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:06.223643    2574 scope.go:117] "RemoveContainer" containerID="29111efb6a9652fc83923d263b237ab6d7c4173e413ff72545f722b94df783ca"
Jul 28 22:53:06.223900 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:53:06.223884    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29111efb6a9652fc83923d263b237ab6d7c4173e413ff72545f722b94df783ca\": container with ID starting with 29111efb6a9652fc83923d263b237ab6d7c4173e413ff72545f722b94df783ca not found: ID does not exist" containerID="29111efb6a9652fc83923d263b237ab6d7c4173e413ff72545f722b94df783ca"
Jul 28 22:53:06.223946 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:06.223909    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29111efb6a9652fc83923d263b237ab6d7c4173e413ff72545f722b94df783ca"} err="failed to get container status \"29111efb6a9652fc83923d263b237ab6d7c4173e413ff72545f722b94df783ca\": rpc error: code = NotFound desc = could not find container \"29111efb6a9652fc83923d263b237ab6d7c4173e413ff72545f722b94df783ca\": container with ID starting with 29111efb6a9652fc83923d263b237ab6d7c4173e413ff72545f722b94df783ca not found: ID does not exist"
Jul 28 22:53:06.234352 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:06.234330    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"]
Jul 28 22:53:06.239026 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:06.239004    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1661d9/sub-test-model-1661d9-kserve-7446f8f689-5llmt"]
Jul 28 22:53:07.495309 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:07.495273    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b090e421-6041-476a-b320-04bc01ed3967" path="/var/lib/kubelet/pods/b090e421-6041-476a-b320-04bc01ed3967/volumes"
Jul 28 22:53:11.016480 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:11.016442    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:11.016984 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:11.016498    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:11.028702 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:11.028676    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:53:11.253431 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:53:11.253403    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:54:33.977093 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:33.977048    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"]
Jul 28 22:54:48.996063 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:48.996020    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4" podUID="fe20fe90-fa5e-483a-b78c-e445c2c098e2" containerName="main" containerID="cri-o://4f5bee513250f1281bca39df776267cdede491b47576c889312d6b3fdbfcdee2" gracePeriod=45
Jul 28 22:54:49.254347 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.254284    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:54:49.286790 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.286760    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-dshm\") pod \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") "
Jul 28 22:54:49.286975 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.286825    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe20fe90-fa5e-483a-b78c-e445c2c098e2-tls-certs\") pod \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") "
Jul 28 22:54:49.286975 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.286862    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-home\") pod \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") "
Jul 28 22:54:49.286975 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.286881    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mh6k\" (UniqueName: \"kubernetes.io/projected/fe20fe90-fa5e-483a-b78c-e445c2c098e2-kube-api-access-6mh6k\") pod \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") "
Jul 28 22:54:49.286975 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.286914    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-model-cache\") pod \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") "
Jul 28 22:54:49.287206 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.287102    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-home" (OuterVolumeSpecName: "home") pod "fe20fe90-fa5e-483a-b78c-e445c2c098e2" (UID: "fe20fe90-fa5e-483a-b78c-e445c2c098e2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:54:49.287300 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.287270    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-home\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:54:49.287300 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.287268    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-model-cache" (OuterVolumeSpecName: "model-cache") pod "fe20fe90-fa5e-483a-b78c-e445c2c098e2" (UID: "fe20fe90-fa5e-483a-b78c-e445c2c098e2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:54:49.289306 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.289275    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-dshm" (OuterVolumeSpecName: "dshm") pod "fe20fe90-fa5e-483a-b78c-e445c2c098e2" (UID: "fe20fe90-fa5e-483a-b78c-e445c2c098e2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:54:49.289432 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.289413    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe20fe90-fa5e-483a-b78c-e445c2c098e2-kube-api-access-6mh6k" (OuterVolumeSpecName: "kube-api-access-6mh6k") pod "fe20fe90-fa5e-483a-b78c-e445c2c098e2" (UID: "fe20fe90-fa5e-483a-b78c-e445c2c098e2"). InnerVolumeSpecName "kube-api-access-6mh6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:54:49.289495 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.289412    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe20fe90-fa5e-483a-b78c-e445c2c098e2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fe20fe90-fa5e-483a-b78c-e445c2c098e2" (UID: "fe20fe90-fa5e-483a-b78c-e445c2c098e2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:54:49.387589 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.387552    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-tmp-dir\") pod \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\" (UID: \"fe20fe90-fa5e-483a-b78c-e445c2c098e2\") "
Jul 28 22:54:49.387761 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.387753    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mh6k\" (UniqueName: \"kubernetes.io/projected/fe20fe90-fa5e-483a-b78c-e445c2c098e2-kube-api-access-6mh6k\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:54:49.387804 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.387764    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-model-cache\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:54:49.387804 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.387773    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-dshm\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:54:49.387804 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.387780    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe20fe90-fa5e-483a-b78c-e445c2c098e2-tls-certs\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:54:49.387894 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.387824    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fe20fe90-fa5e-483a-b78c-e445c2c098e2" (UID: "fe20fe90-fa5e-483a-b78c-e445c2c098e2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:54:49.488447 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.488415    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe20fe90-fa5e-483a-b78c-e445c2c098e2-tmp-dir\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:54:49.724038 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.724002    2574 generic.go:356] "Generic (PLEG): container finished" podID="fe20fe90-fa5e-483a-b78c-e445c2c098e2" containerID="4f5bee513250f1281bca39df776267cdede491b47576c889312d6b3fdbfcdee2" exitCode=0
Jul 28 22:54:49.724206 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.724048    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"
Jul 28 22:54:49.724206 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.724074    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4" event={"ID":"fe20fe90-fa5e-483a-b78c-e445c2c098e2","Type":"ContainerDied","Data":"4f5bee513250f1281bca39df776267cdede491b47576c889312d6b3fdbfcdee2"}
Jul 28 22:54:49.724206 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.724113    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4" event={"ID":"fe20fe90-fa5e-483a-b78c-e445c2c098e2","Type":"ContainerDied","Data":"0eef2b30a06c76e36ee0f8b00dc4c45a1b502cf6a5ad1066aacdda77ba299425"}
Jul 28 22:54:49.724206 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.724134    2574 scope.go:117] "RemoveContainer" containerID="4f5bee513250f1281bca39df776267cdede491b47576c889312d6b3fdbfcdee2"
Jul 28 22:54:49.732779 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.732759    2574 scope.go:117] "RemoveContainer" containerID="4f5bee513250f1281bca39df776267cdede491b47576c889312d6b3fdbfcdee2"
Jul 28 22:54:49.733045 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:54:49.733025    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f5bee513250f1281bca39df776267cdede491b47576c889312d6b3fdbfcdee2\": container with ID starting with 4f5bee513250f1281bca39df776267cdede491b47576c889312d6b3fdbfcdee2 not found: ID does not exist" containerID="4f5bee513250f1281bca39df776267cdede491b47576c889312d6b3fdbfcdee2"
Jul 28 22:54:49.733092 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.733054    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f5bee513250f1281bca39df776267cdede491b47576c889312d6b3fdbfcdee2"} err="failed to get container status \"4f5bee513250f1281bca39df776267cdede491b47576c889312d6b3fdbfcdee2\": rpc error: code = NotFound desc = could not find container \"4f5bee513250f1281bca39df776267cdede491b47576c889312d6b3fdbfcdee2\": container with ID starting with 4f5bee513250f1281bca39df776267cdede491b47576c889312d6b3fdbfcdee2 not found: ID does not exist"
Jul 28 22:54:49.738165 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.738143    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"]
Jul 28 22:54:49.741470 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:49.741447    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ae4fa9/rate-test-model-ae4fa9-kserve-6888cdfc9-8qbx4"]
Jul 28 22:54:51.494467 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:51.494431    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe20fe90-fa5e-483a-b78c-e445c2c098e2" path="/var/lib/kubelet/pods/fe20fe90-fa5e-483a-b78c-e445c2c098e2/volumes"
Jul 28 22:54:57.697657 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:57.697623    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq"]
Jul 28 22:54:57.698119 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:57.697983    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2047c26e-b66e-4626-9399-1c38d82669b8" containerName="main"
Jul 28 22:54:57.698119 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:57.697994    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2047c26e-b66e-4626-9399-1c38d82669b8" containerName="main"
Jul 28 22:54:57.698119 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:57.698004    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b090e421-6041-476a-b320-04bc01ed3967" containerName="main"
Jul 28 22:54:57.698119 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:57.698009    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b090e421-6041-476a-b320-04bc01ed3967" containerName="main"
Jul 28 22:54:57.698119 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:57.698025    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe20fe90-fa5e-483a-b78c-e445c2c098e2" containerName="main"
Jul 28 22:54:57.698119 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:57.698030    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe20fe90-fa5e-483a-b78c-e445c2c098e2" containerName="main"
Jul 28 22:54:57.698119 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:57.698083    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe20fe90-fa5e-483a-b78c-e445c2c098e2" containerName="main"
Jul 28 22:54:57.698119 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:57.698089    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b090e421-6041-476a-b320-04bc01ed3967" containerName="main"
Jul 28 22:54:57.698119 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:57.698094    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2047c26e-b66e-4626-9399-1c38d82669b8" containerName="main"
Jul 28 22:54:57.702720 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:57.702699    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq"
Jul 28 22:54:57.704342 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:57.704322    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-ff4fa1-serving-cert\""
Jul 28 22:54:57.711987 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:57.711956    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq"]
Jul 28 22:54:57.754712 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:57.754677    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/140af177-5044-4bd1-9912-1c7982c4f77e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq\" (UID: \"140af177-5044-4bd1-9912-1c7982c4f77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq"
Jul 28 22:54:57.754712 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:57.754718    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmr8f\" (UniqueName: \"kubernetes.io/projected/140af177-5044-4bd1-9912-1c7982c4f77e-kube-api-access-hmr8f\") pod \"maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq\" (UID: \"140af177-5044-4bd1-9912-1c7982c4f77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq"
Jul 28 22:54:57.754927 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:57.754746    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/140af177-5044-4bd1-9912-1c7982c4f77e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq\" (UID: \"140af177-5044-4bd1-9912-1c7982c4f77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq"
Jul 28 22:54:57.855273 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:57.855214    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/140af177-5044-4bd1-9912-1c7982c4f77e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq\" (UID: \"140af177-5044-4bd1-9912-1c7982c4f77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq"
Jul 28 22:54:57.855273 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:57.855277    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmr8f\" (UniqueName: \"kubernetes.io/projected/140af177-5044-4bd1-9912-1c7982c4f77e-kube-api-access-hmr8f\") pod \"maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq\" (UID: \"140af177-5044-4bd1-9912-1c7982c4f77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq"
Jul 28 22:54:57.855495 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:57.855300    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/140af177-5044-4bd1-9912-1c7982c4f77e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq\" (UID: \"140af177-5044-4bd1-9912-1c7982c4f77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq"
Jul 28 22:54:57.855495 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:54:57.855371    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-serving-cert: secret "maas-api-e2e-ipp-a-ff4fa1-serving-cert" not found
Jul 28 22:54:57.855495 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:54:57.855440    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/140af177-5044-4bd1-9912-1c7982c4f77e-maas-api-tls podName:140af177-5044-4bd1-9912-1c7982c4f77e nodeName:}" failed. No retries permitted until 2026-07-28 22:54:58.355424516 +0000 UTC m=+3027.339567113 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/140af177-5044-4bd1-9912-1c7982c4f77e-maas-api-tls") pod "maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq" (UID: "140af177-5044-4bd1-9912-1c7982c4f77e") : secret "maas-api-e2e-ipp-a-ff4fa1-serving-cert" not found
Jul 28 22:54:57.856008 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:57.855972    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/140af177-5044-4bd1-9912-1c7982c4f77e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq\" (UID: \"140af177-5044-4bd1-9912-1c7982c4f77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq"
Jul 28 22:54:57.867602 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:57.867575    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmr8f\" (UniqueName: \"kubernetes.io/projected/140af177-5044-4bd1-9912-1c7982c4f77e-kube-api-access-hmr8f\") pod \"maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq\" (UID: \"140af177-5044-4bd1-9912-1c7982c4f77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq"
Jul 28 22:54:58.359701 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:58.359662    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/140af177-5044-4bd1-9912-1c7982c4f77e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq\" (UID: \"140af177-5044-4bd1-9912-1c7982c4f77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq"
Jul 28 22:54:58.362111 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:58.362089    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/140af177-5044-4bd1-9912-1c7982c4f77e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq\" (UID: \"140af177-5044-4bd1-9912-1c7982c4f77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq"
Jul 28 22:54:58.614474 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:58.614391    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq"
Jul 28 22:54:58.758441 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:58.758417    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq"]
Jul 28 22:54:58.761079 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:54:58.761048    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod140af177_5044_4bd1_9912_1c7982c4f77e.slice/crio-ba4f7ae352281881e1522cc59def45834a64d1c2d8e5c4766b80b704fa3519e3 WatchSource:0}: Error finding container ba4f7ae352281881e1522cc59def45834a64d1c2d8e5c4766b80b704fa3519e3: Status 404 returned error can't find the container with id ba4f7ae352281881e1522cc59def45834a64d1c2d8e5c4766b80b704fa3519e3
Jul 28 22:54:58.762482 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:58.762462    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 22:54:58.772896 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:58.772869    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq" event={"ID":"140af177-5044-4bd1-9912-1c7982c4f77e","Type":"ContainerStarted","Data":"ba4f7ae352281881e1522cc59def45834a64d1c2d8e5c4766b80b704fa3519e3"}
Jul 28 22:54:59.779719 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:59.779681    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq" event={"ID":"140af177-5044-4bd1-9912-1c7982c4f77e","Type":"ContainerStarted","Data":"7f9fe39be47ae562346dec58b65d08bf93eac34bbcef72567ec3509540b1e97b"}
Jul 28 22:54:59.780109 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:59.779741    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq"
Jul 28 22:54:59.801056 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:54:59.800951    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq" podStartSLOduration=2.377155671 podStartE2EDuration="2.800934762s" totalImagesPullingTime="423.779091ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:54:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:54:59.798727636 +0000 UTC m=+3028.782870254" watchObservedRunningTime="2026-07-28 22:54:59.800934762 +0000 UTC m=+3028.785077381"
Jul 28 22:55:05.788704 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:55:05.788674    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq"
Jul 28 22:55:26.745398 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:55:26.745279    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj"]
Jul 28 22:55:26.748948 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:55:26.748921    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj"
Jul 28 22:55:26.750788 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:55:26.750764    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-056c8a-serving-cert\""
Jul 28 22:55:26.758641 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:55:26.758616    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj"]
Jul 28 22:55:26.824196 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:55:26.824154    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkg9f\" (UniqueName: \"kubernetes.io/projected/7fd88638-f7eb-4c2c-84da-2501414fe643-kube-api-access-fkg9f\") pod \"maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj\" (UID: \"7fd88638-f7eb-4c2c-84da-2501414fe643\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj"
Jul 28 22:55:26.824401 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:55:26.824205    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fd88638-f7eb-4c2c-84da-2501414fe643-maas-api-tls\") pod \"maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj\" (UID: \"7fd88638-f7eb-4c2c-84da-2501414fe643\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj"
Jul 28 22:55:26.824401 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:55:26.824312    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fd88638-f7eb-4c2c-84da-2501414fe643-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj\" (UID: \"7fd88638-f7eb-4c2c-84da-2501414fe643\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj"
Jul 28 22:55:26.924913 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:55:26.924875    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fd88638-f7eb-4c2c-84da-2501414fe643-maas-api-tls\") pod \"maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj\" (UID: \"7fd88638-f7eb-4c2c-84da-2501414fe643\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj"
Jul 28 22:55:26.925114 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:55:26.924943    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fd88638-f7eb-4c2c-84da-2501414fe643-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj\" (UID: \"7fd88638-f7eb-4c2c-84da-2501414fe643\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj"
Jul 28 22:55:26.925114 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:55:26.925000    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkg9f\" (UniqueName: \"kubernetes.io/projected/7fd88638-f7eb-4c2c-84da-2501414fe643-kube-api-access-fkg9f\") pod \"maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj\" (UID: \"7fd88638-f7eb-4c2c-84da-2501414fe643\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj"
Jul 28 22:55:26.925648 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:55:26.925625    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fd88638-f7eb-4c2c-84da-2501414fe643-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj\" (UID: \"7fd88638-f7eb-4c2c-84da-2501414fe643\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj"
Jul 28 22:55:26.927363 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:55:26.927342    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fd88638-f7eb-4c2c-84da-2501414fe643-maas-api-tls\") pod \"maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj\" (UID: \"7fd88638-f7eb-4c2c-84da-2501414fe643\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj"
Jul 28 22:55:26.938348 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:55:26.938321    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkg9f\" (UniqueName: \"kubernetes.io/projected/7fd88638-f7eb-4c2c-84da-2501414fe643-kube-api-access-fkg9f\") pod \"maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj\" (UID: \"7fd88638-f7eb-4c2c-84da-2501414fe643\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj"
Jul 28 22:55:27.061211 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:55:27.061117    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj"
Jul 28 22:55:27.200666 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:55:27.200641    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj"]
Jul 28 22:55:27.203415 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:55:27.203381    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fd88638_f7eb_4c2c_84da_2501414fe643.slice/crio-ed958f78da4d6d10a473f3cf00b6d0d7040eb22f025c0bab0b5dc05a956e1be4 WatchSource:0}: Error finding container ed958f78da4d6d10a473f3cf00b6d0d7040eb22f025c0bab0b5dc05a956e1be4: Status 404 returned error can't find the container with id ed958f78da4d6d10a473f3cf00b6d0d7040eb22f025c0bab0b5dc05a956e1be4
Jul 28 22:55:27.914139 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:55:27.914035    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj" event={"ID":"7fd88638-f7eb-4c2c-84da-2501414fe643","Type":"ContainerStarted","Data":"5de512b9be8345785d8736cd8d84909ab689bb9971df554aa7ee965315fafe06"}
Jul 28 22:55:27.914139 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:55:27.914092    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj" event={"ID":"7fd88638-f7eb-4c2c-84da-2501414fe643","Type":"ContainerStarted","Data":"ed958f78da4d6d10a473f3cf00b6d0d7040eb22f025c0bab0b5dc05a956e1be4"}
Jul 28 22:55:27.914139 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:55:27.914116    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj"
Jul 28 22:55:27.929611 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:55:27.929544    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj" podStartSLOduration=1.5853006870000002 podStartE2EDuration="1.929524167s" totalImagesPullingTime="344.22348ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:55:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:55:27.928652754 +0000 UTC m=+3056.912795373" watchObservedRunningTime="2026-07-28 22:55:27.929524167 +0000 UTC m=+3056.913666786"
Jul 28 22:55:33.923973 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:55:33.923941    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj"
Jul 28 22:56:00.605690 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.605653    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"]
Jul 28 22:56:00.609281 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.609255    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:00.612032 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.612006    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-ff4fa1\"/\"kube-root-ca.crt\""
Jul 28 22:56:00.612271 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.612252    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-ff4fa1\"/\"openshift-service-ca.crt\""
Jul 28 22:56:00.612386 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.612314    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-ff4fa1\"/\"ipp-route-ff4fa1-kserve-self-signed-certs\""
Jul 28 22:56:00.612510 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.612489    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-ff4fa1\"/\"default-dockercfg-nkffj\""
Jul 28 22:56:00.618790 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.618764    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"]
Jul 28 22:56:00.739743 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.739698    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-tmp-dir\") pod \"ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") " pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:00.739932 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.739752    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-dshm\") pod \"ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") " pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:00.739932 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.739829    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-home\") pod \"ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") " pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:00.739932 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.739856    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvhnt\" (UniqueName: \"kubernetes.io/projected/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-kube-api-access-mvhnt\") pod \"ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") " pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:00.740070 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.739974    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-tls-certs\") pod \"ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") " pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:00.740070 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.740047    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-model-cache\") pod \"ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") " pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:00.841442 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.841412    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-tls-certs\") pod \"ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") " pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:00.841600 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.841469    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-model-cache\") pod \"ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") " pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:00.841600 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.841501    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-tmp-dir\") pod \"ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") " pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:00.841600 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.841518    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-dshm\") pod \"ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") " pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:00.841734 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.841594    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-home\") pod \"ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") " pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:00.841734 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.841629    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvhnt\" (UniqueName: \"kubernetes.io/projected/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-kube-api-access-mvhnt\") pod \"ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") " pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:00.842035 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.842005    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-tmp-dir\") pod \"ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") " pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:00.842157 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.842013    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-home\") pod \"ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") " pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:00.842213 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.842156    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-model-cache\") pod \"ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") " pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:00.844048 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.844026    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-dshm\") pod \"ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") " pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:00.844360 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.844337    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-tls-certs\") pod \"ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") " pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:00.848895 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.848873    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvhnt\" (UniqueName: \"kubernetes.io/projected/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-kube-api-access-mvhnt\") pod \"ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") " pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:00.920015 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:00.919918    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:01.065733 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:01.065707    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"]
Jul 28 22:56:01.067883 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:56:01.067858    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ffbc8ff_97bc_4d94_99ec_7c4fd4727061.slice/crio-7ab957119c388e465ea04c9caf768dbc666af6bf275f031a10f00a8546e3b222 WatchSource:0}: Error finding container 7ab957119c388e465ea04c9caf768dbc666af6bf275f031a10f00a8546e3b222: Status 404 returned error can't find the container with id 7ab957119c388e465ea04c9caf768dbc666af6bf275f031a10f00a8546e3b222
Jul 28 22:56:02.073798 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:02.073758    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl" event={"ID":"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061","Type":"ContainerStarted","Data":"9bc1cf8073f651a24465735075ba526519bbc9542502373d4f72773683f05b29"}
Jul 28 22:56:02.074207 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:02.073812    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl" event={"ID":"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061","Type":"ContainerStarted","Data":"7ab957119c388e465ea04c9caf768dbc666af6bf275f031a10f00a8546e3b222"}
Jul 28 22:56:02.093431 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:02.093377    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl" podStartSLOduration=2.09336212 podStartE2EDuration="2.09336212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:56:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:56:02.090862223 +0000 UTC m=+3091.075004863" watchObservedRunningTime="2026-07-28 22:56:02.09336212 +0000 UTC m=+3091.077504737"
Jul 28 22:56:10.920366 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:10.920337    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:10.920366 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:10.920385    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:10.932581 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:10.932557    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:11.127519 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:11.127489    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:56:49.402528 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.402497    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"]
Jul 28 22:56:49.406222 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.406202    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.408223 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.408200    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-5b3501-openshift-default-dockercfg-cb8t7\""
Jul 28 22:56:49.421879 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.421845    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"]
Jul 28 22:56:49.486034 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.485994    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-data\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.486252 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.486049    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-workload-certs\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.486252 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.486069    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-podinfo\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.486252 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.486109    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bp2w\" (UniqueName: \"kubernetes.io/projected/e77f7753-2d42-44e2-a0b3-ae63876b18af-kube-api-access-7bp2w\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.486252 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.486128    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-envoy\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.486252 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.486142    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-workload-socket\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.486252 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.486170    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-credential-socket\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.486252 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.486219    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-token\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.486252 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.486250    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e77f7753-2d42-44e2-a0b3-ae63876b18af-istiod-ca-cert\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.587318 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.587282    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-token\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.587318 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.587321    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e77f7753-2d42-44e2-a0b3-ae63876b18af-istiod-ca-cert\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.587568 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.587386    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-data\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.587568 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.587418    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-workload-certs\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.587568 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.587442    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-podinfo\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.587568 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.587474    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bp2w\" (UniqueName: \"kubernetes.io/projected/e77f7753-2d42-44e2-a0b3-ae63876b18af-kube-api-access-7bp2w\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.587568 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.587501    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-envoy\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.587568 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.587525    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-workload-socket\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.587568 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.587549    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-credential-socket\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.587968 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.587861    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-workload-certs\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.588043 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.588018    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-credential-socket\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.588187 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.588161    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-data\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.588330 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.588304    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e77f7753-2d42-44e2-a0b3-ae63876b18af-istiod-ca-cert\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.588502 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.588309    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-workload-socket\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.590592 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.590566    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-envoy\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.590911 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.590886    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-podinfo\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.594884 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.594863    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-token\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.595423 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.595394    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bp2w\" (UniqueName: \"kubernetes.io/projected/e77f7753-2d42-44e2-a0b3-ae63876b18af-kube-api-access-7bp2w\") pod \"e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") " pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.716290 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.716226    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:49.858697 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.858656    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"]
Jul 28 22:56:49.860931 ip-10-0-143-69 kubenswrapper[2574]: W0728 22:56:49.860904    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode77f7753_2d42_44e2_a0b3_ae63876b18af.slice/crio-f09d497b068083537605f562dd81df61b4460b19c7abdd17ece89e6dc8cd0cc8 WatchSource:0}: Error finding container f09d497b068083537605f562dd81df61b4460b19c7abdd17ece89e6dc8cd0cc8: Status 404 returned error can't find the container with id f09d497b068083537605f562dd81df61b4460b19c7abdd17ece89e6dc8cd0cc8
Jul 28 22:56:49.863132 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.863097    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:56:49.863223 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.863161    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:56:49.863223 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:49.863191    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:56:50.309997 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:50.309960    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng" event={"ID":"e77f7753-2d42-44e2-a0b3-ae63876b18af","Type":"ContainerStarted","Data":"78a79ae3091c51acfd6bb6c902a70e8c72a6ce05a4fa9f45419b2375ce749299"}
Jul 28 22:56:50.310188 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:50.310017    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng" event={"ID":"e77f7753-2d42-44e2-a0b3-ae63876b18af","Type":"ContainerStarted","Data":"f09d497b068083537605f562dd81df61b4460b19c7abdd17ece89e6dc8cd0cc8"}
Jul 28 22:56:50.329993 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:50.329948    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng" podStartSLOduration=1.329931234 podStartE2EDuration="1.329931234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:56:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:56:50.328884291 +0000 UTC m=+3139.313026910" watchObservedRunningTime="2026-07-28 22:56:50.329931234 +0000 UTC m=+3139.314073854"
Jul 28 22:56:50.716941 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:50.716903    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:50.722130 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:50.722106    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:51.315682 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:51.315650    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:56:51.316631 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:56:51.316611    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:57:05.390490 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:05.390406    2574 generic.go:356] "Generic (PLEG): container finished" podID="e77f7753-2d42-44e2-a0b3-ae63876b18af" containerID="78a79ae3091c51acfd6bb6c902a70e8c72a6ce05a4fa9f45419b2375ce749299" exitCode=0
Jul 28 22:57:05.390490 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:05.390444    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng" event={"ID":"e77f7753-2d42-44e2-a0b3-ae63876b18af","Type":"ContainerDied","Data":"78a79ae3091c51acfd6bb6c902a70e8c72a6ce05a4fa9f45419b2375ce749299"}
Jul 28 22:57:05.390937 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:05.390819    2574 scope.go:117] "RemoveContainer" containerID="78a79ae3091c51acfd6bb6c902a70e8c72a6ce05a4fa9f45419b2375ce749299"
Jul 28 22:57:05.391896 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:05.391865    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:57:05.391949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:05.391937    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:57:05.391984 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:05.391974    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:57:06.316414 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:06.316372    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:57:06.397186 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:06.397140    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng" event={"ID":"e77f7753-2d42-44e2-a0b3-ae63876b18af","Type":"ContainerStarted","Data":"86de750792c4dd3f5e8ef223e25e6df880fff81d5be5022145779dcfcc120ed7"}
Jul 28 22:57:06.716843 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:06.716812    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:57:06.717961 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:06.717935    2574 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Jul 28 22:57:06.718080 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:06.717983    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng" podUID="e77f7753-2d42-44e2-a0b3-ae63876b18af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Jul 28 22:57:07.717569 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:07.717535    2574 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Jul 28 22:57:07.717933 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:07.717590    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng" podUID="e77f7753-2d42-44e2-a0b3-ae63876b18af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Jul 28 22:57:08.717342 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:08.717306    2574 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Jul 28 22:57:08.717544 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:08.717403    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng" podUID="e77f7753-2d42-44e2-a0b3-ae63876b18af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Jul 28 22:57:09.717028 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:09.716990    2574 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Jul 28 22:57:09.717450 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:09.717054    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng" podUID="e77f7753-2d42-44e2-a0b3-ae63876b18af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Jul 28 22:57:10.721259 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:10.721203    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:57:10.721693 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:10.721511    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:57:10.722280 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:10.722264    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:57:29.649595 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:29.649560    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"]
Jul 28 22:57:29.650176 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:29.649776    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng" podUID="e77f7753-2d42-44e2-a0b3-ae63876b18af" containerName="istio-proxy" containerID="cri-o://86de750792c4dd3f5e8ef223e25e6df880fff81d5be5022145779dcfcc120ed7" gracePeriod=30
Jul 28 22:57:34.908164 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:34.908138    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:57:35.020712 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.020676    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-workload-certs\") pod \"e77f7753-2d42-44e2-a0b3-ae63876b18af\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") "
Jul 28 22:57:35.020894 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.020729    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e77f7753-2d42-44e2-a0b3-ae63876b18af-istiod-ca-cert\") pod \"e77f7753-2d42-44e2-a0b3-ae63876b18af\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") "
Jul 28 22:57:35.020894 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.020763    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-workload-socket\") pod \"e77f7753-2d42-44e2-a0b3-ae63876b18af\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") "
Jul 28 22:57:35.020894 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.020787    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-token\") pod \"e77f7753-2d42-44e2-a0b3-ae63876b18af\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") "
Jul 28 22:57:35.020894 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.020811    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-envoy\") pod \"e77f7753-2d42-44e2-a0b3-ae63876b18af\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") "
Jul 28 22:57:35.020894 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.020838    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-credential-socket\") pod \"e77f7753-2d42-44e2-a0b3-ae63876b18af\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") "
Jul 28 22:57:35.020894 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.020889    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bp2w\" (UniqueName: \"kubernetes.io/projected/e77f7753-2d42-44e2-a0b3-ae63876b18af-kube-api-access-7bp2w\") pod \"e77f7753-2d42-44e2-a0b3-ae63876b18af\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") "
Jul 28 22:57:35.021220 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.020919    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-podinfo\") pod \"e77f7753-2d42-44e2-a0b3-ae63876b18af\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") "
Jul 28 22:57:35.021220 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.020981    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-data\") pod \"e77f7753-2d42-44e2-a0b3-ae63876b18af\" (UID: \"e77f7753-2d42-44e2-a0b3-ae63876b18af\") "
Jul 28 22:57:35.021220 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.021117    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e77f7753-2d42-44e2-a0b3-ae63876b18af" (UID: "e77f7753-2d42-44e2-a0b3-ae63876b18af"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:57:35.021220 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.021199    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e77f7753-2d42-44e2-a0b3-ae63876b18af" (UID: "e77f7753-2d42-44e2-a0b3-ae63876b18af"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:57:35.021481 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.021291    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e77f7753-2d42-44e2-a0b3-ae63876b18af-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e77f7753-2d42-44e2-a0b3-ae63876b18af" (UID: "e77f7753-2d42-44e2-a0b3-ae63876b18af"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:57:35.021481 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.021335    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e77f7753-2d42-44e2-a0b3-ae63876b18af" (UID: "e77f7753-2d42-44e2-a0b3-ae63876b18af"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:57:35.021577 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.021483    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-workload-certs\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:57:35.021577 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.021505    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e77f7753-2d42-44e2-a0b3-ae63876b18af-istiod-ca-cert\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:57:35.021577 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.021520    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-workload-socket\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:57:35.021577 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.021532    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-credential-socket\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:57:35.021925 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.021902    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-data" (OuterVolumeSpecName: "istio-data") pod "e77f7753-2d42-44e2-a0b3-ae63876b18af" (UID: "e77f7753-2d42-44e2-a0b3-ae63876b18af"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:57:35.023169 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.023144    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e77f7753-2d42-44e2-a0b3-ae63876b18af" (UID: "e77f7753-2d42-44e2-a0b3-ae63876b18af"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:57:35.023305 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.023200    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-token" (OuterVolumeSpecName: "istio-token") pod "e77f7753-2d42-44e2-a0b3-ae63876b18af" (UID: "e77f7753-2d42-44e2-a0b3-ae63876b18af"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:57:35.023462 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.023437    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e77f7753-2d42-44e2-a0b3-ae63876b18af" (UID: "e77f7753-2d42-44e2-a0b3-ae63876b18af"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 22:57:35.023545 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.023485    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e77f7753-2d42-44e2-a0b3-ae63876b18af-kube-api-access-7bp2w" (OuterVolumeSpecName: "kube-api-access-7bp2w") pod "e77f7753-2d42-44e2-a0b3-ae63876b18af" (UID: "e77f7753-2d42-44e2-a0b3-ae63876b18af"). InnerVolumeSpecName "kube-api-access-7bp2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:57:35.122511 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.122476    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-token\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:57:35.122511 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.122505    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-envoy\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:57:35.122511 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.122515    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bp2w\" (UniqueName: \"kubernetes.io/projected/e77f7753-2d42-44e2-a0b3-ae63876b18af-kube-api-access-7bp2w\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:57:35.122748 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.122524    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-podinfo\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:57:35.122748 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.122533    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e77f7753-2d42-44e2-a0b3-ae63876b18af-istio-data\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:57:35.552914 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.552878    2574 generic.go:356] "Generic (PLEG): container finished" podID="e77f7753-2d42-44e2-a0b3-ae63876b18af" containerID="86de750792c4dd3f5e8ef223e25e6df880fff81d5be5022145779dcfcc120ed7" exitCode=0
Jul 28 22:57:35.553101 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.552935    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"
Jul 28 22:57:35.553101 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.552956    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng" event={"ID":"e77f7753-2d42-44e2-a0b3-ae63876b18af","Type":"ContainerDied","Data":"86de750792c4dd3f5e8ef223e25e6df880fff81d5be5022145779dcfcc120ed7"}
Jul 28 22:57:35.553101 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.553029    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng" event={"ID":"e77f7753-2d42-44e2-a0b3-ae63876b18af","Type":"ContainerDied","Data":"f09d497b068083537605f562dd81df61b4460b19c7abdd17ece89e6dc8cd0cc8"}
Jul 28 22:57:35.553101 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.553059    2574 scope.go:117] "RemoveContainer" containerID="86de750792c4dd3f5e8ef223e25e6df880fff81d5be5022145779dcfcc120ed7"
Jul 28 22:57:35.562589 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.562566    2574 scope.go:117] "RemoveContainer" containerID="78a79ae3091c51acfd6bb6c902a70e8c72a6ce05a4fa9f45419b2375ce749299"
Jul 28 22:57:35.568477 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.568450    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"]
Jul 28 22:57:35.571659 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.571642    2574 scope.go:117] "RemoveContainer" containerID="86de750792c4dd3f5e8ef223e25e6df880fff81d5be5022145779dcfcc120ed7"
Jul 28 22:57:35.571724 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.571696    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5b3501-openshift-default-6c7bfd866d-zppng"]
Jul 28 22:57:35.571913 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:57:35.571895    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86de750792c4dd3f5e8ef223e25e6df880fff81d5be5022145779dcfcc120ed7\": container with ID starting with 86de750792c4dd3f5e8ef223e25e6df880fff81d5be5022145779dcfcc120ed7 not found: ID does not exist" containerID="86de750792c4dd3f5e8ef223e25e6df880fff81d5be5022145779dcfcc120ed7"
Jul 28 22:57:35.571958 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.571921    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86de750792c4dd3f5e8ef223e25e6df880fff81d5be5022145779dcfcc120ed7"} err="failed to get container status \"86de750792c4dd3f5e8ef223e25e6df880fff81d5be5022145779dcfcc120ed7\": rpc error: code = NotFound desc = could not find container \"86de750792c4dd3f5e8ef223e25e6df880fff81d5be5022145779dcfcc120ed7\": container with ID starting with 86de750792c4dd3f5e8ef223e25e6df880fff81d5be5022145779dcfcc120ed7 not found: ID does not exist"
Jul 28 22:57:35.571958 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.571939    2574 scope.go:117] "RemoveContainer" containerID="78a79ae3091c51acfd6bb6c902a70e8c72a6ce05a4fa9f45419b2375ce749299"
Jul 28 22:57:35.572207 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:57:35.572190    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78a79ae3091c51acfd6bb6c902a70e8c72a6ce05a4fa9f45419b2375ce749299\": container with ID starting with 78a79ae3091c51acfd6bb6c902a70e8c72a6ce05a4fa9f45419b2375ce749299 not found: ID does not exist" containerID="78a79ae3091c51acfd6bb6c902a70e8c72a6ce05a4fa9f45419b2375ce749299"
Jul 28 22:57:35.572283 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:35.572212    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78a79ae3091c51acfd6bb6c902a70e8c72a6ce05a4fa9f45419b2375ce749299"} err="failed to get container status \"78a79ae3091c51acfd6bb6c902a70e8c72a6ce05a4fa9f45419b2375ce749299\": rpc error: code = NotFound desc = could not find container \"78a79ae3091c51acfd6bb6c902a70e8c72a6ce05a4fa9f45419b2375ce749299\": container with ID starting with 78a79ae3091c51acfd6bb6c902a70e8c72a6ce05a4fa9f45419b2375ce749299 not found: ID does not exist"
Jul 28 22:57:37.494362 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:37.494331    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e77f7753-2d42-44e2-a0b3-ae63876b18af" path="/var/lib/kubelet/pods/e77f7753-2d42-44e2-a0b3-ae63876b18af/volumes"
Jul 28 22:57:48.830133 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:48.830092    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq"]
Jul 28 22:57:48.830566 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:48.830341    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq" podUID="140af177-5044-4bd1-9912-1c7982c4f77e" containerName="maas-api" containerID="cri-o://7f9fe39be47ae562346dec58b65d08bf93eac34bbcef72567ec3509540b1e97b" gracePeriod=30
Jul 28 22:57:49.101069 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:49.101044    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq"
Jul 28 22:57:49.257454 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:49.257413    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/140af177-5044-4bd1-9912-1c7982c4f77e-maas-api-tls\") pod \"140af177-5044-4bd1-9912-1c7982c4f77e\" (UID: \"140af177-5044-4bd1-9912-1c7982c4f77e\") "
Jul 28 22:57:49.257648 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:49.257508    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmr8f\" (UniqueName: \"kubernetes.io/projected/140af177-5044-4bd1-9912-1c7982c4f77e-kube-api-access-hmr8f\") pod \"140af177-5044-4bd1-9912-1c7982c4f77e\" (UID: \"140af177-5044-4bd1-9912-1c7982c4f77e\") "
Jul 28 22:57:49.257724 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:49.257662    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/140af177-5044-4bd1-9912-1c7982c4f77e-openshift-service-ca\") pod \"140af177-5044-4bd1-9912-1c7982c4f77e\" (UID: \"140af177-5044-4bd1-9912-1c7982c4f77e\") "
Jul 28 22:57:49.258025 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:49.257998    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/140af177-5044-4bd1-9912-1c7982c4f77e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "140af177-5044-4bd1-9912-1c7982c4f77e" (UID: "140af177-5044-4bd1-9912-1c7982c4f77e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:57:49.259503 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:49.259476    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/140af177-5044-4bd1-9912-1c7982c4f77e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "140af177-5044-4bd1-9912-1c7982c4f77e" (UID: "140af177-5044-4bd1-9912-1c7982c4f77e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:57:49.259599 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:49.259503    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/140af177-5044-4bd1-9912-1c7982c4f77e-kube-api-access-hmr8f" (OuterVolumeSpecName: "kube-api-access-hmr8f") pod "140af177-5044-4bd1-9912-1c7982c4f77e" (UID: "140af177-5044-4bd1-9912-1c7982c4f77e"). InnerVolumeSpecName "kube-api-access-hmr8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:57:49.358901 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:49.358822    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/140af177-5044-4bd1-9912-1c7982c4f77e-openshift-service-ca\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:57:49.358901 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:49.358852    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/140af177-5044-4bd1-9912-1c7982c4f77e-maas-api-tls\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:57:49.358901 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:49.358862    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmr8f\" (UniqueName: \"kubernetes.io/projected/140af177-5044-4bd1-9912-1c7982c4f77e-kube-api-access-hmr8f\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:57:49.622737 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:49.622638    2574 generic.go:356] "Generic (PLEG): container finished" podID="140af177-5044-4bd1-9912-1c7982c4f77e" containerID="7f9fe39be47ae562346dec58b65d08bf93eac34bbcef72567ec3509540b1e97b" exitCode=0
Jul 28 22:57:49.622737 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:49.622686    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq"
Jul 28 22:57:49.622737 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:49.622707    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq" event={"ID":"140af177-5044-4bd1-9912-1c7982c4f77e","Type":"ContainerDied","Data":"7f9fe39be47ae562346dec58b65d08bf93eac34bbcef72567ec3509540b1e97b"}
Jul 28 22:57:49.623005 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:49.622769    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq" event={"ID":"140af177-5044-4bd1-9912-1c7982c4f77e","Type":"ContainerDied","Data":"ba4f7ae352281881e1522cc59def45834a64d1c2d8e5c4766b80b704fa3519e3"}
Jul 28 22:57:49.623005 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:49.622793    2574 scope.go:117] "RemoveContainer" containerID="7f9fe39be47ae562346dec58b65d08bf93eac34bbcef72567ec3509540b1e97b"
Jul 28 22:57:49.631242 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:49.631213    2574 scope.go:117] "RemoveContainer" containerID="7f9fe39be47ae562346dec58b65d08bf93eac34bbcef72567ec3509540b1e97b"
Jul 28 22:57:49.631517 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:57:49.631499    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f9fe39be47ae562346dec58b65d08bf93eac34bbcef72567ec3509540b1e97b\": container with ID starting with 7f9fe39be47ae562346dec58b65d08bf93eac34bbcef72567ec3509540b1e97b not found: ID does not exist" containerID="7f9fe39be47ae562346dec58b65d08bf93eac34bbcef72567ec3509540b1e97b"
Jul 28 22:57:49.631589 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:49.631525    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f9fe39be47ae562346dec58b65d08bf93eac34bbcef72567ec3509540b1e97b"} err="failed to get container status \"7f9fe39be47ae562346dec58b65d08bf93eac34bbcef72567ec3509540b1e97b\": rpc error: code = NotFound desc = could not find container \"7f9fe39be47ae562346dec58b65d08bf93eac34bbcef72567ec3509540b1e97b\": container with ID starting with 7f9fe39be47ae562346dec58b65d08bf93eac34bbcef72567ec3509540b1e97b not found: ID does not exist"
Jul 28 22:57:49.637774 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:49.637747    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq"]
Jul 28 22:57:49.642859 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:49.642837    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-ff4fa1-7b7b875967-jqsqq"]
Jul 28 22:57:51.493828 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:51.493794    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="140af177-5044-4bd1-9912-1c7982c4f77e" path="/var/lib/kubelet/pods/140af177-5044-4bd1-9912-1c7982c4f77e/volumes"
Jul 28 22:57:54.414143 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:57:54.414100    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"]
Jul 28 22:58:09.432950 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.432885    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl" podUID="6ffbc8ff-97bc-4d94-99ec-7c4fd4727061" containerName="main" containerID="cri-o://9bc1cf8073f651a24465735075ba526519bbc9542502373d4f72773683f05b29" gracePeriod=45
Jul 28 22:58:09.696317 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.696295    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:58:09.718405 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.718375    2574 generic.go:356] "Generic (PLEG): container finished" podID="6ffbc8ff-97bc-4d94-99ec-7c4fd4727061" containerID="9bc1cf8073f651a24465735075ba526519bbc9542502373d4f72773683f05b29" exitCode=0
Jul 28 22:58:09.718534 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.718416    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"
Jul 28 22:58:09.718534 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.718452    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl" event={"ID":"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061","Type":"ContainerDied","Data":"9bc1cf8073f651a24465735075ba526519bbc9542502373d4f72773683f05b29"}
Jul 28 22:58:09.718534 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.718494    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl" event={"ID":"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061","Type":"ContainerDied","Data":"7ab957119c388e465ea04c9caf768dbc666af6bf275f031a10f00a8546e3b222"}
Jul 28 22:58:09.718534 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.718511    2574 scope.go:117] "RemoveContainer" containerID="9bc1cf8073f651a24465735075ba526519bbc9542502373d4f72773683f05b29"
Jul 28 22:58:09.726264 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.726245    2574 scope.go:117] "RemoveContainer" containerID="9bc1cf8073f651a24465735075ba526519bbc9542502373d4f72773683f05b29"
Jul 28 22:58:09.726498 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:58:09.726480    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bc1cf8073f651a24465735075ba526519bbc9542502373d4f72773683f05b29\": container with ID starting with 9bc1cf8073f651a24465735075ba526519bbc9542502373d4f72773683f05b29 not found: ID does not exist" containerID="9bc1cf8073f651a24465735075ba526519bbc9542502373d4f72773683f05b29"
Jul 28 22:58:09.726544 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.726507    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bc1cf8073f651a24465735075ba526519bbc9542502373d4f72773683f05b29"} err="failed to get container status \"9bc1cf8073f651a24465735075ba526519bbc9542502373d4f72773683f05b29\": rpc error: code = NotFound desc = could not find container \"9bc1cf8073f651a24465735075ba526519bbc9542502373d4f72773683f05b29\": container with ID starting with 9bc1cf8073f651a24465735075ba526519bbc9542502373d4f72773683f05b29 not found: ID does not exist"
Jul 28 22:58:09.743879 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.743847    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-dshm\") pod \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") "
Jul 28 22:58:09.744035 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.743920    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvhnt\" (UniqueName: \"kubernetes.io/projected/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-kube-api-access-mvhnt\") pod \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") "
Jul 28 22:58:09.744035 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.744002    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-home\") pod \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") "
Jul 28 22:58:09.744165 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.744036    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-model-cache\") pod \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") "
Jul 28 22:58:09.744165 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.744068    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-tmp-dir\") pod \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") "
Jul 28 22:58:09.744165 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.744115    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-tls-certs\") pod \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\" (UID: \"6ffbc8ff-97bc-4d94-99ec-7c4fd4727061\") "
Jul 28 22:58:09.744554 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.744490    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6ffbc8ff-97bc-4d94-99ec-7c4fd4727061" (UID: "6ffbc8ff-97bc-4d94-99ec-7c4fd4727061"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:58:09.744554 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.744509    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-model-cache" (OuterVolumeSpecName: "model-cache") pod "6ffbc8ff-97bc-4d94-99ec-7c4fd4727061" (UID: "6ffbc8ff-97bc-4d94-99ec-7c4fd4727061"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:58:09.744744 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.744602    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-home" (OuterVolumeSpecName: "home") pod "6ffbc8ff-97bc-4d94-99ec-7c4fd4727061" (UID: "6ffbc8ff-97bc-4d94-99ec-7c4fd4727061"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:58:09.746467 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.746444    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-dshm" (OuterVolumeSpecName: "dshm") pod "6ffbc8ff-97bc-4d94-99ec-7c4fd4727061" (UID: "6ffbc8ff-97bc-4d94-99ec-7c4fd4727061"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:58:09.746629 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.746611    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-kube-api-access-mvhnt" (OuterVolumeSpecName: "kube-api-access-mvhnt") pod "6ffbc8ff-97bc-4d94-99ec-7c4fd4727061" (UID: "6ffbc8ff-97bc-4d94-99ec-7c4fd4727061"). InnerVolumeSpecName "kube-api-access-mvhnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:58:09.746803 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.746769    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6ffbc8ff-97bc-4d94-99ec-7c4fd4727061" (UID: "6ffbc8ff-97bc-4d94-99ec-7c4fd4727061"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:58:09.845409 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.845378    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-home\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:58:09.845409 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.845405    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-model-cache\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:58:09.845409 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.845415    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-tmp-dir\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:58:09.845653 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.845422    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-tls-certs\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:58:09.845653 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.845430    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-dshm\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:58:09.845653 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:09.845437    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvhnt\" (UniqueName: \"kubernetes.io/projected/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061-kube-api-access-mvhnt\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:58:10.037768 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:10.037739    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"]
Jul 28 22:58:10.041030 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:10.041005    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-ff4fa1/ipp-route-ff4fa1-kserve-5ccdbccdc5-g24xl"]
Jul 28 22:58:11.498937 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:11.498893    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ffbc8ff-97bc-4d94-99ec-7c4fd4727061" path="/var/lib/kubelet/pods/6ffbc8ff-97bc-4d94-99ec-7c4fd4727061/volumes"
Jul 28 22:58:49.445824 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:49.445786    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj"]
Jul 28 22:58:49.446403 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:49.446030    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj" podUID="7fd88638-f7eb-4c2c-84da-2501414fe643" containerName="maas-api" containerID="cri-o://5de512b9be8345785d8736cd8d84909ab689bb9971df554aa7ee965315fafe06" gracePeriod=30
Jul 28 22:58:49.731866 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:49.731839    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj"
Jul 28 22:58:49.805068 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:49.805028    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fd88638-f7eb-4c2c-84da-2501414fe643-openshift-service-ca\") pod \"7fd88638-f7eb-4c2c-84da-2501414fe643\" (UID: \"7fd88638-f7eb-4c2c-84da-2501414fe643\") "
Jul 28 22:58:49.805273 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:49.805089    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkg9f\" (UniqueName: \"kubernetes.io/projected/7fd88638-f7eb-4c2c-84da-2501414fe643-kube-api-access-fkg9f\") pod \"7fd88638-f7eb-4c2c-84da-2501414fe643\" (UID: \"7fd88638-f7eb-4c2c-84da-2501414fe643\") "
Jul 28 22:58:49.805273 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:49.805131    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fd88638-f7eb-4c2c-84da-2501414fe643-maas-api-tls\") pod \"7fd88638-f7eb-4c2c-84da-2501414fe643\" (UID: \"7fd88638-f7eb-4c2c-84da-2501414fe643\") "
Jul 28 22:58:49.805477 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:49.805456    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fd88638-f7eb-4c2c-84da-2501414fe643-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7fd88638-f7eb-4c2c-84da-2501414fe643" (UID: "7fd88638-f7eb-4c2c-84da-2501414fe643"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:58:49.807211 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:49.807185    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fd88638-f7eb-4c2c-84da-2501414fe643-kube-api-access-fkg9f" (OuterVolumeSpecName: "kube-api-access-fkg9f") pod "7fd88638-f7eb-4c2c-84da-2501414fe643" (UID: "7fd88638-f7eb-4c2c-84da-2501414fe643"). InnerVolumeSpecName "kube-api-access-fkg9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:58:49.807290 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:49.807246    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fd88638-f7eb-4c2c-84da-2501414fe643-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7fd88638-f7eb-4c2c-84da-2501414fe643" (UID: "7fd88638-f7eb-4c2c-84da-2501414fe643"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:58:49.905256 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:49.905196    2574 generic.go:356] "Generic (PLEG): container finished" podID="7fd88638-f7eb-4c2c-84da-2501414fe643" containerID="5de512b9be8345785d8736cd8d84909ab689bb9971df554aa7ee965315fafe06" exitCode=0
Jul 28 22:58:49.905414 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:49.905268    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj"
Jul 28 22:58:49.905414 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:49.905272    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj" event={"ID":"7fd88638-f7eb-4c2c-84da-2501414fe643","Type":"ContainerDied","Data":"5de512b9be8345785d8736cd8d84909ab689bb9971df554aa7ee965315fafe06"}
Jul 28 22:58:49.905414 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:49.905333    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj" event={"ID":"7fd88638-f7eb-4c2c-84da-2501414fe643","Type":"ContainerDied","Data":"ed958f78da4d6d10a473f3cf00b6d0d7040eb22f025c0bab0b5dc05a956e1be4"}
Jul 28 22:58:49.905414 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:49.905364    2574 scope.go:117] "RemoveContainer" containerID="5de512b9be8345785d8736cd8d84909ab689bb9971df554aa7ee965315fafe06"
Jul 28 22:58:49.905844 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:49.905820    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fd88638-f7eb-4c2c-84da-2501414fe643-maas-api-tls\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:58:49.905949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:49.905848    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fd88638-f7eb-4c2c-84da-2501414fe643-openshift-service-ca\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:58:49.905949 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:49.905860    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkg9f\" (UniqueName: \"kubernetes.io/projected/7fd88638-f7eb-4c2c-84da-2501414fe643-kube-api-access-fkg9f\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:58:49.914523 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:49.914508    2574 scope.go:117] "RemoveContainer" containerID="5de512b9be8345785d8736cd8d84909ab689bb9971df554aa7ee965315fafe06"
Jul 28 22:58:49.914761 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:58:49.914745    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5de512b9be8345785d8736cd8d84909ab689bb9971df554aa7ee965315fafe06\": container with ID starting with 5de512b9be8345785d8736cd8d84909ab689bb9971df554aa7ee965315fafe06 not found: ID does not exist" containerID="5de512b9be8345785d8736cd8d84909ab689bb9971df554aa7ee965315fafe06"
Jul 28 22:58:49.914820 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:49.914768    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5de512b9be8345785d8736cd8d84909ab689bb9971df554aa7ee965315fafe06"} err="failed to get container status \"5de512b9be8345785d8736cd8d84909ab689bb9971df554aa7ee965315fafe06\": rpc error: code = NotFound desc = could not find container \"5de512b9be8345785d8736cd8d84909ab689bb9971df554aa7ee965315fafe06\": container with ID starting with 5de512b9be8345785d8736cd8d84909ab689bb9971df554aa7ee965315fafe06 not found: ID does not exist"
Jul 28 22:58:49.925303 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:49.925280    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj"]
Jul 28 22:58:49.930619 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:49.930599    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-056c8a-68c78f695-lsxmj"]
Jul 28 22:58:51.499052 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:58:51.499018    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fd88638-f7eb-4c2c-84da-2501414fe643" path="/var/lib/kubelet/pods/7fd88638-f7eb-4c2c-84da-2501414fe643/volumes"
Jul 28 22:59:07.014477 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:07.014440    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf"]
Jul 28 22:59:07.014931 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:07.014691    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf" podUID="b86255cd-4ef7-474b-a335-a581d64b7cda" containerName="maas-api" containerID="cri-o://6e8fee90f528177ecd57c21bee41a03c6e7dde2272b8ec6ea33d54a01de34002" gracePeriod=30
Jul 28 22:59:07.259033 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:07.259009    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf"
Jul 28 22:59:07.367074 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:07.366984    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b86255cd-4ef7-474b-a335-a581d64b7cda-openshift-service-ca\") pod \"b86255cd-4ef7-474b-a335-a581d64b7cda\" (UID: \"b86255cd-4ef7-474b-a335-a581d64b7cda\") "
Jul 28 22:59:07.367074 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:07.367054    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b86255cd-4ef7-474b-a335-a581d64b7cda-maas-api-tls\") pod \"b86255cd-4ef7-474b-a335-a581d64b7cda\" (UID: \"b86255cd-4ef7-474b-a335-a581d64b7cda\") "
Jul 28 22:59:07.367326 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:07.367156    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgwrh\" (UniqueName: \"kubernetes.io/projected/b86255cd-4ef7-474b-a335-a581d64b7cda-kube-api-access-vgwrh\") pod \"b86255cd-4ef7-474b-a335-a581d64b7cda\" (UID: \"b86255cd-4ef7-474b-a335-a581d64b7cda\") "
Jul 28 22:59:07.367417 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:07.367390    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b86255cd-4ef7-474b-a335-a581d64b7cda-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b86255cd-4ef7-474b-a335-a581d64b7cda" (UID: "b86255cd-4ef7-474b-a335-a581d64b7cda"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:59:07.369055 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:07.369027    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b86255cd-4ef7-474b-a335-a581d64b7cda-kube-api-access-vgwrh" (OuterVolumeSpecName: "kube-api-access-vgwrh") pod "b86255cd-4ef7-474b-a335-a581d64b7cda" (UID: "b86255cd-4ef7-474b-a335-a581d64b7cda"). InnerVolumeSpecName "kube-api-access-vgwrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:59:07.369215 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:07.369121    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b86255cd-4ef7-474b-a335-a581d64b7cda-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b86255cd-4ef7-474b-a335-a581d64b7cda" (UID: "b86255cd-4ef7-474b-a335-a581d64b7cda"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:59:07.468700 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:07.468668    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgwrh\" (UniqueName: \"kubernetes.io/projected/b86255cd-4ef7-474b-a335-a581d64b7cda-kube-api-access-vgwrh\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:59:07.468700 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:07.468696    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b86255cd-4ef7-474b-a335-a581d64b7cda-openshift-service-ca\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:59:07.468700 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:07.468705    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b86255cd-4ef7-474b-a335-a581d64b7cda-maas-api-tls\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:59:07.997617 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:07.997587    2574 generic.go:356] "Generic (PLEG): container finished" podID="b86255cd-4ef7-474b-a335-a581d64b7cda" containerID="6e8fee90f528177ecd57c21bee41a03c6e7dde2272b8ec6ea33d54a01de34002" exitCode=0
Jul 28 22:59:07.997804 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:07.997635    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf"
Jul 28 22:59:07.997804 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:07.997663    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf" event={"ID":"b86255cd-4ef7-474b-a335-a581d64b7cda","Type":"ContainerDied","Data":"6e8fee90f528177ecd57c21bee41a03c6e7dde2272b8ec6ea33d54a01de34002"}
Jul 28 22:59:07.997804 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:07.997702    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf" event={"ID":"b86255cd-4ef7-474b-a335-a581d64b7cda","Type":"ContainerDied","Data":"3d4f3f8b332d2fe516a84046335f7f3ad00b9e9fd93ddd2f99886441ef2db236"}
Jul 28 22:59:07.997804 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:07.997723    2574 scope.go:117] "RemoveContainer" containerID="6e8fee90f528177ecd57c21bee41a03c6e7dde2272b8ec6ea33d54a01de34002"
Jul 28 22:59:08.006124 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:08.006105    2574 scope.go:117] "RemoveContainer" containerID="6e8fee90f528177ecd57c21bee41a03c6e7dde2272b8ec6ea33d54a01de34002"
Jul 28 22:59:08.006416 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:59:08.006397    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e8fee90f528177ecd57c21bee41a03c6e7dde2272b8ec6ea33d54a01de34002\": container with ID starting with 6e8fee90f528177ecd57c21bee41a03c6e7dde2272b8ec6ea33d54a01de34002 not found: ID does not exist" containerID="6e8fee90f528177ecd57c21bee41a03c6e7dde2272b8ec6ea33d54a01de34002"
Jul 28 22:59:08.006478 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:08.006425    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e8fee90f528177ecd57c21bee41a03c6e7dde2272b8ec6ea33d54a01de34002"} err="failed to get container status \"6e8fee90f528177ecd57c21bee41a03c6e7dde2272b8ec6ea33d54a01de34002\": rpc error: code = NotFound desc = could not find container \"6e8fee90f528177ecd57c21bee41a03c6e7dde2272b8ec6ea33d54a01de34002\": container with ID starting with 6e8fee90f528177ecd57c21bee41a03c6e7dde2272b8ec6ea33d54a01de34002 not found: ID does not exist"
Jul 28 22:59:08.012908 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:08.012886    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf"]
Jul 28 22:59:08.016534 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:08.016513    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ae4fa9-86466b7c89-2s8lf"]
Jul 28 22:59:09.493769 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:09.493737    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b86255cd-4ef7-474b-a335-a581d64b7cda" path="/var/lib/kubelet/pods/b86255cd-4ef7-474b-a335-a581d64b7cda/volumes"
Jul 28 22:59:13.836566 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:13.836495    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"]
Jul 28 22:59:14.025320 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:14.025280    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj" podUID="c474c0ba-3644-4872-9921-4bb7d49856a8" containerName="istio-proxy" containerID="cri-o://004dd808bd5e27cd52246245c3c76ec4990508c3837d29d4daba073a98fdfb13" gracePeriod=30
Jul 28 22:59:19.286185 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.286162    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:59:19.380317 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.380220    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-token\") pod \"c474c0ba-3644-4872-9921-4bb7d49856a8\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") "
Jul 28 22:59:19.380317 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.380279    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-workload-certs\") pod \"c474c0ba-3644-4872-9921-4bb7d49856a8\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") "
Jul 28 22:59:19.380317 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.380300    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8g8p\" (UniqueName: \"kubernetes.io/projected/c474c0ba-3644-4872-9921-4bb7d49856a8-kube-api-access-r8g8p\") pod \"c474c0ba-3644-4872-9921-4bb7d49856a8\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") "
Jul 28 22:59:19.380554 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.380337    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-workload-socket\") pod \"c474c0ba-3644-4872-9921-4bb7d49856a8\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") "
Jul 28 22:59:19.380554 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.380372    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-data\") pod \"c474c0ba-3644-4872-9921-4bb7d49856a8\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") "
Jul 28 22:59:19.380554 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.380415    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-envoy\") pod \"c474c0ba-3644-4872-9921-4bb7d49856a8\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") "
Jul 28 22:59:19.380554 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.380446    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-podinfo\") pod \"c474c0ba-3644-4872-9921-4bb7d49856a8\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") "
Jul 28 22:59:19.380554 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.380466    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c474c0ba-3644-4872-9921-4bb7d49856a8-istiod-ca-cert\") pod \"c474c0ba-3644-4872-9921-4bb7d49856a8\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") "
Jul 28 22:59:19.380554 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.380542    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-credential-socket\") pod \"c474c0ba-3644-4872-9921-4bb7d49856a8\" (UID: \"c474c0ba-3644-4872-9921-4bb7d49856a8\") "
Jul 28 22:59:19.380856 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.380678    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c474c0ba-3644-4872-9921-4bb7d49856a8" (UID: "c474c0ba-3644-4872-9921-4bb7d49856a8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:59:19.380856 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.380689    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c474c0ba-3644-4872-9921-4bb7d49856a8" (UID: "c474c0ba-3644-4872-9921-4bb7d49856a8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:59:19.380961 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.380856    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-workload-certs\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:59:19.380961 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.380874    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-workload-socket\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:59:19.381074 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.381005    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c474c0ba-3644-4872-9921-4bb7d49856a8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c474c0ba-3644-4872-9921-4bb7d49856a8" (UID: "c474c0ba-3644-4872-9921-4bb7d49856a8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:59:19.381132 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.381101    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c474c0ba-3644-4872-9921-4bb7d49856a8" (UID: "c474c0ba-3644-4872-9921-4bb7d49856a8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:59:19.381508 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.381480    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-data" (OuterVolumeSpecName: "istio-data") pod "c474c0ba-3644-4872-9921-4bb7d49856a8" (UID: "c474c0ba-3644-4872-9921-4bb7d49856a8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:59:19.382636 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.382609    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c474c0ba-3644-4872-9921-4bb7d49856a8" (UID: "c474c0ba-3644-4872-9921-4bb7d49856a8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:59:19.382636 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.382620    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c474c0ba-3644-4872-9921-4bb7d49856a8-kube-api-access-r8g8p" (OuterVolumeSpecName: "kube-api-access-r8g8p") pod "c474c0ba-3644-4872-9921-4bb7d49856a8" (UID: "c474c0ba-3644-4872-9921-4bb7d49856a8"). InnerVolumeSpecName "kube-api-access-r8g8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:59:19.382636 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.382609    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-token" (OuterVolumeSpecName: "istio-token") pod "c474c0ba-3644-4872-9921-4bb7d49856a8" (UID: "c474c0ba-3644-4872-9921-4bb7d49856a8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:59:19.382806 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.382699    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c474c0ba-3644-4872-9921-4bb7d49856a8" (UID: "c474c0ba-3644-4872-9921-4bb7d49856a8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 22:59:19.481859 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.481831    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-data\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:59:19.481859 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.481854    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-envoy\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:59:19.481859 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.481863    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-podinfo\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:59:19.482073 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.481872    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c474c0ba-3644-4872-9921-4bb7d49856a8-istiod-ca-cert\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:59:19.482073 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.481880    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c474c0ba-3644-4872-9921-4bb7d49856a8-credential-socket\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:59:19.482073 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.481889    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c474c0ba-3644-4872-9921-4bb7d49856a8-istio-token\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:59:19.482073 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:19.481896    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8g8p\" (UniqueName: \"kubernetes.io/projected/c474c0ba-3644-4872-9921-4bb7d49856a8-kube-api-access-r8g8p\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:59:20.054577 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:20.054545    2574 generic.go:356] "Generic (PLEG): container finished" podID="c474c0ba-3644-4872-9921-4bb7d49856a8" containerID="004dd808bd5e27cd52246245c3c76ec4990508c3837d29d4daba073a98fdfb13" exitCode=0
Jul 28 22:59:20.054747 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:20.054589    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj" event={"ID":"c474c0ba-3644-4872-9921-4bb7d49856a8","Type":"ContainerDied","Data":"004dd808bd5e27cd52246245c3c76ec4990508c3837d29d4daba073a98fdfb13"}
Jul 28 22:59:20.054747 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:20.054601    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"
Jul 28 22:59:20.054747 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:20.054632    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj" event={"ID":"c474c0ba-3644-4872-9921-4bb7d49856a8","Type":"ContainerDied","Data":"15216259c1ab8d9b1b4a4f6815eb378686715abde807a1127477a715c74e7290"}
Jul 28 22:59:20.054747 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:20.054653    2574 scope.go:117] "RemoveContainer" containerID="004dd808bd5e27cd52246245c3c76ec4990508c3837d29d4daba073a98fdfb13"
Jul 28 22:59:20.063357 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:20.063253    2574 scope.go:117] "RemoveContainer" containerID="f18ae5ac9d5b8f41f1331e12bdc28b95107841c6dff4acb91d7c03b68f831312"
Jul 28 22:59:20.069839 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:20.069814    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"]
Jul 28 22:59:20.071563 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:20.071516    2574 scope.go:117] "RemoveContainer" containerID="004dd808bd5e27cd52246245c3c76ec4990508c3837d29d4daba073a98fdfb13"
Jul 28 22:59:20.071827 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:59:20.071806    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"004dd808bd5e27cd52246245c3c76ec4990508c3837d29d4daba073a98fdfb13\": container with ID starting with 004dd808bd5e27cd52246245c3c76ec4990508c3837d29d4daba073a98fdfb13 not found: ID does not exist" containerID="004dd808bd5e27cd52246245c3c76ec4990508c3837d29d4daba073a98fdfb13"
Jul 28 22:59:20.071915 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:20.071846    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"004dd808bd5e27cd52246245c3c76ec4990508c3837d29d4daba073a98fdfb13"} err="failed to get container status \"004dd808bd5e27cd52246245c3c76ec4990508c3837d29d4daba073a98fdfb13\": rpc error: code = NotFound desc = could not find container \"004dd808bd5e27cd52246245c3c76ec4990508c3837d29d4daba073a98fdfb13\": container with ID starting with 004dd808bd5e27cd52246245c3c76ec4990508c3837d29d4daba073a98fdfb13 not found: ID does not exist"
Jul 28 22:59:20.071915 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:20.071864    2574 scope.go:117] "RemoveContainer" containerID="f18ae5ac9d5b8f41f1331e12bdc28b95107841c6dff4acb91d7c03b68f831312"
Jul 28 22:59:20.072120 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:59:20.072103    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f18ae5ac9d5b8f41f1331e12bdc28b95107841c6dff4acb91d7c03b68f831312\": container with ID starting with f18ae5ac9d5b8f41f1331e12bdc28b95107841c6dff4acb91d7c03b68f831312 not found: ID does not exist" containerID="f18ae5ac9d5b8f41f1331e12bdc28b95107841c6dff4acb91d7c03b68f831312"
Jul 28 22:59:20.072170 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:20.072124    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f18ae5ac9d5b8f41f1331e12bdc28b95107841c6dff4acb91d7c03b68f831312"} err="failed to get container status \"f18ae5ac9d5b8f41f1331e12bdc28b95107841c6dff4acb91d7c03b68f831312\": rpc error: code = NotFound desc = could not find container \"f18ae5ac9d5b8f41f1331e12bdc28b95107841c6dff4acb91d7c03b68f831312\": container with ID starting with f18ae5ac9d5b8f41f1331e12bdc28b95107841c6dff4acb91d7c03b68f831312 not found: ID does not exist"
Jul 28 22:59:20.073357 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:20.073339    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-ae4fa9-openshift-default-7494bcc476-zp8pj"]
Jul 28 22:59:21.493439 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:21.493408    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c474c0ba-3644-4872-9921-4bb7d49856a8" path="/var/lib/kubelet/pods/c474c0ba-3644-4872-9921-4bb7d49856a8/volumes"
Jul 28 22:59:24.656885 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:24.656848    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99"]
Jul 28 22:59:24.657372 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:24.657114    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99" podUID="e8cff19b-7cca-4051-a2a6-2ae828c9868d" containerName="maas-api" containerID="cri-o://590eae72a35d53fd48343d11d87f26b2238aade90ec57ae8236bdeef63b21245" gracePeriod=30
Jul 28 22:59:24.902583 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:24.902562    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99"
Jul 28 22:59:25.033578 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:25.033549    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8cff19b-7cca-4051-a2a6-2ae828c9868d-maas-api-tls\") pod \"e8cff19b-7cca-4051-a2a6-2ae828c9868d\" (UID: \"e8cff19b-7cca-4051-a2a6-2ae828c9868d\") "
Jul 28 22:59:25.033767 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:25.033615    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8cff19b-7cca-4051-a2a6-2ae828c9868d-openshift-service-ca\") pod \"e8cff19b-7cca-4051-a2a6-2ae828c9868d\" (UID: \"e8cff19b-7cca-4051-a2a6-2ae828c9868d\") "
Jul 28 22:59:25.033767 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:25.033673    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjpzk\" (UniqueName: \"kubernetes.io/projected/e8cff19b-7cca-4051-a2a6-2ae828c9868d-kube-api-access-bjpzk\") pod \"e8cff19b-7cca-4051-a2a6-2ae828c9868d\" (UID: \"e8cff19b-7cca-4051-a2a6-2ae828c9868d\") "
Jul 28 22:59:25.033984 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:25.033957    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8cff19b-7cca-4051-a2a6-2ae828c9868d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e8cff19b-7cca-4051-a2a6-2ae828c9868d" (UID: "e8cff19b-7cca-4051-a2a6-2ae828c9868d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:59:25.035829 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:25.035794    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8cff19b-7cca-4051-a2a6-2ae828c9868d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e8cff19b-7cca-4051-a2a6-2ae828c9868d" (UID: "e8cff19b-7cca-4051-a2a6-2ae828c9868d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:59:25.035977 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:25.035885    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8cff19b-7cca-4051-a2a6-2ae828c9868d-kube-api-access-bjpzk" (OuterVolumeSpecName: "kube-api-access-bjpzk") pod "e8cff19b-7cca-4051-a2a6-2ae828c9868d" (UID: "e8cff19b-7cca-4051-a2a6-2ae828c9868d"). InnerVolumeSpecName "kube-api-access-bjpzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:59:25.085551 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:25.085519    2574 generic.go:356] "Generic (PLEG): container finished" podID="e8cff19b-7cca-4051-a2a6-2ae828c9868d" containerID="590eae72a35d53fd48343d11d87f26b2238aade90ec57ae8236bdeef63b21245" exitCode=0
Jul 28 22:59:25.085695 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:25.085566    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99"
Jul 28 22:59:25.085695 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:25.085597    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99" event={"ID":"e8cff19b-7cca-4051-a2a6-2ae828c9868d","Type":"ContainerDied","Data":"590eae72a35d53fd48343d11d87f26b2238aade90ec57ae8236bdeef63b21245"}
Jul 28 22:59:25.085695 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:25.085648    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99" event={"ID":"e8cff19b-7cca-4051-a2a6-2ae828c9868d","Type":"ContainerDied","Data":"070ef923a69c258aae724b4089f854dd547a0d275e705962d2524b219f2c2d03"}
Jul 28 22:59:25.085695 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:25.085671    2574 scope.go:117] "RemoveContainer" containerID="590eae72a35d53fd48343d11d87f26b2238aade90ec57ae8236bdeef63b21245"
Jul 28 22:59:25.094271 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:25.094255    2574 scope.go:117] "RemoveContainer" containerID="590eae72a35d53fd48343d11d87f26b2238aade90ec57ae8236bdeef63b21245"
Jul 28 22:59:25.094522 ip-10-0-143-69 kubenswrapper[2574]: E0728 22:59:25.094502    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"590eae72a35d53fd48343d11d87f26b2238aade90ec57ae8236bdeef63b21245\": container with ID starting with 590eae72a35d53fd48343d11d87f26b2238aade90ec57ae8236bdeef63b21245 not found: ID does not exist" containerID="590eae72a35d53fd48343d11d87f26b2238aade90ec57ae8236bdeef63b21245"
Jul 28 22:59:25.094603 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:25.094528    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"590eae72a35d53fd48343d11d87f26b2238aade90ec57ae8236bdeef63b21245"} err="failed to get container status \"590eae72a35d53fd48343d11d87f26b2238aade90ec57ae8236bdeef63b21245\": rpc error: code = NotFound desc = could not find container \"590eae72a35d53fd48343d11d87f26b2238aade90ec57ae8236bdeef63b21245\": container with ID starting with 590eae72a35d53fd48343d11d87f26b2238aade90ec57ae8236bdeef63b21245 not found: ID does not exist"
Jul 28 22:59:25.107106 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:25.107083    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99"]
Jul 28 22:59:25.109788 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:25.109760    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-1661d9-7d6999469-nvs99"]
Jul 28 22:59:25.134323 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:25.134302    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8cff19b-7cca-4051-a2a6-2ae828c9868d-openshift-service-ca\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:59:25.134323 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:25.134321    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjpzk\" (UniqueName: \"kubernetes.io/projected/e8cff19b-7cca-4051-a2a6-2ae828c9868d-kube-api-access-bjpzk\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:59:25.134323 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:25.134332    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8cff19b-7cca-4051-a2a6-2ae828c9868d-maas-api-tls\") on node \"ip-10-0-143-69.ec2.internal\" DevicePath \"\""
Jul 28 22:59:25.494955 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:25.494925    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8cff19b-7cca-4051-a2a6-2ae828c9868d" path="/var/lib/kubelet/pods/e8cff19b-7cca-4051-a2a6-2ae828c9868d/volumes"
Jul 28 22:59:32.232560 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:32.232525    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7658b4fb4d-ht9hh_9a821774-325d-46ae-81bc-457b729317fb/authorino/0.log"
Jul 28 22:59:37.639450 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:37.639399    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-cp5wh_21574045-310f-4d71-9427-477488ae7bbb/manager/0.log"
Jul 28 22:59:37.762071 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:37.762039    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jvrt4_846482be-f263-409b-a5f2-a290e50d2cb8/manager/0.log"
Jul 28 22:59:37.888355 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:37.888324    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-796f547645-h6r2g_86d33c6f-e796-4e0f-984e-a1703cd034a2/manager/0.log"
Jul 28 22:59:38.006247 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:38.006215    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-5nmvv_f8b44918-780b-4433-8136-5bf279e14dd5/server/0.log"
Jul 28 22:59:38.132073 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:38.132044    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-xpb8v_e0162e33-5b05-4e2e-a9a3-c58e95c401dd/manager/0.log"
Jul 28 22:59:38.243414 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:38.243388    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d45996775-fzq6v_b03a58fd-5c5b-44fd-b6c2-cc9a6a455178/copy-manifests/0.log"
Jul 28 22:59:38.250069 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:38.250034    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d45996775-fzq6v_b03a58fd-5c5b-44fd-b6c2-cc9a6a455178/manager/0.log"
Jul 28 22:59:38.868101 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:38.868071    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-8dd88f89-pj89b_6c814da8-6e2d-476f-b0b9-d3d2eaaa02b8/maas-api/0.log"
Jul 28 22:59:40.608143 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:40.608112    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7658b4fb4d-ht9hh_9a821774-325d-46ae-81bc-457b729317fb/authorino/0.log"
Jul 28 22:59:40.727728 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:40.727695    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-xf4vp_ac4a98b1-7dd9-4269-ae2c-af949745aa15/manager/0.log"
Jul 28 22:59:40.847781 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:40.847752    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-8jsfk_46c2b945-8fe4-4ed0-9776-fd5336b32cda/manager/0.log"
Jul 28 22:59:41.086311 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:41.086281    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-dzr7c_12d8cd98-7bc6-4991-b4c1-bd5ce9a4592b/registry-server/0.log"
Jul 28 22:59:41.213348 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:41.213317    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-hxw7b_51300b59-611c-4d9d-a350-120f6e105afc/manager/0.log"
Jul 28 22:59:41.448383 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:41.448356    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-4ns4b_8559d00a-827c-4feb-ad81-b1e3f7fcdedc/manager/0.log"
Jul 28 22:59:41.811276 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:41.811177    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799crd6zz_1262efc1-7845-4349-8425-0609954af5c6/istio-proxy/0.log"
Jul 28 22:59:42.036386 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:42.036356    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cd5d84f76-9pg4h_cc63bade-aba2-4894-97a9-d4fc058fad23/kube-auth-proxy/0.log"
Jul 28 22:59:42.266828 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:42.266791    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-xfw54_a55b9c60-64aa-4a57-82d7-6309caa50034/istio-proxy/1.log"
Jul 28 22:59:42.387072 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:42.387036    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-m6qn9_1afb94a8-a577-4cab-93f7-e3cab182cd83/payload-pre-processing/0.log"
Jul 28 22:59:43.192794 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:43.192763    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-hvb8r_05fb5c53-d53e-42fd-b037-966389f058e6/main/0.log"
Jul 28 22:59:43.526268 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:43.526171    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-ktkrn_2f2cb5ec-81c7-4bc3-98a4-99853ad7057f/main/0.log"
Jul 28 22:59:52.372283 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:52.372247    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qw7gj_6bbf80a3-a5e2-49a5-ad56-a21747cc4a16/global-pull-secret-syncer/0.log"
Jul 28 22:59:52.554374 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:52.554346    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bzhk8_c6e1d829-065c-4516-a1cf-456e7d3ff8c1/konnectivity-agent/0.log"
Jul 28 22:59:52.627596 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:52.627513    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-69.ec2.internal_7cc3a9f08dccc44aae468666893de9e2/haproxy/0.log"
Jul 28 22:59:57.687255 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:57.687155    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7658b4fb4d-ht9hh_9a821774-325d-46ae-81bc-457b729317fb/authorino/0.log"
Jul 28 22:59:57.721105 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:57.721077    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-xf4vp_ac4a98b1-7dd9-4269-ae2c-af949745aa15/manager/0.log"
Jul 28 22:59:57.763142 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:57.763119    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-8jsfk_46c2b945-8fe4-4ed0-9776-fd5336b32cda/manager/0.log"
Jul 28 22:59:57.864123 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:57.864096    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-dzr7c_12d8cd98-7bc6-4991-b4c1-bd5ce9a4592b/registry-server/0.log"
Jul 28 22:59:58.115305 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:58.115273    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-hxw7b_51300b59-611c-4d9d-a350-120f6e105afc/manager/0.log"
Jul 28 22:59:58.258413 ip-10-0-143-69 kubenswrapper[2574]: I0728 22:59:58.258378    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-4ns4b_8559d00a-827c-4feb-ad81-b1e3f7fcdedc/manager/0.log"
Jul 28 23:00:00.548206 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:00.548171    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b5d92_0e15a45f-7342-41b6-a8d3-3d3b7e2945ae/node-exporter/0.log"
Jul 28 23:00:00.572591 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:00.572560    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b5d92_0e15a45f-7342-41b6-a8d3-3d3b7e2945ae/kube-rbac-proxy/0.log"
Jul 28 23:00:00.598270 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:00.598225    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b5d92_0e15a45f-7342-41b6-a8d3-3d3b7e2945ae/init-textfile/0.log"
Jul 28 23:00:03.042038 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.041999    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s"]
Jul 28 23:00:03.042432 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042349    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c474c0ba-3644-4872-9921-4bb7d49856a8" containerName="istio-proxy"
Jul 28 23:00:03.042432 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042361    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c474c0ba-3644-4872-9921-4bb7d49856a8" containerName="istio-proxy"
Jul 28 23:00:03.042432 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042369    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8cff19b-7cca-4051-a2a6-2ae828c9868d" containerName="maas-api"
Jul 28 23:00:03.042432 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042375    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8cff19b-7cca-4051-a2a6-2ae828c9868d" containerName="maas-api"
Jul 28 23:00:03.042432 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042382    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b86255cd-4ef7-474b-a335-a581d64b7cda" containerName="maas-api"
Jul 28 23:00:03.042432 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042389    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b86255cd-4ef7-474b-a335-a581d64b7cda" containerName="maas-api"
Jul 28 23:00:03.042432 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042397    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c474c0ba-3644-4872-9921-4bb7d49856a8" containerName="istio-proxy"
Jul 28 23:00:03.042432 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042402    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c474c0ba-3644-4872-9921-4bb7d49856a8" containerName="istio-proxy"
Jul 28 23:00:03.042432 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042409    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="140af177-5044-4bd1-9912-1c7982c4f77e" containerName="maas-api"
Jul 28 23:00:03.042432 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042415    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="140af177-5044-4bd1-9912-1c7982c4f77e" containerName="maas-api"
Jul 28 23:00:03.042432 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042426    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e77f7753-2d42-44e2-a0b3-ae63876b18af" containerName="istio-proxy"
Jul 28 23:00:03.042432 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042432    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e77f7753-2d42-44e2-a0b3-ae63876b18af" containerName="istio-proxy"
Jul 28 23:00:03.042794 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042443    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ffbc8ff-97bc-4d94-99ec-7c4fd4727061" containerName="main"
Jul 28 23:00:03.042794 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042449    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ffbc8ff-97bc-4d94-99ec-7c4fd4727061" containerName="main"
Jul 28 23:00:03.042794 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042456    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fd88638-f7eb-4c2c-84da-2501414fe643" containerName="maas-api"
Jul 28 23:00:03.042794 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042461    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fd88638-f7eb-4c2c-84da-2501414fe643" containerName="maas-api"
Jul 28 23:00:03.042794 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042513    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fd88638-f7eb-4c2c-84da-2501414fe643" containerName="maas-api"
Jul 28 23:00:03.042794 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042519    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c474c0ba-3644-4872-9921-4bb7d49856a8" containerName="istio-proxy"
Jul 28 23:00:03.042794 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042526    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ffbc8ff-97bc-4d94-99ec-7c4fd4727061" containerName="main"
Jul 28 23:00:03.042794 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042532    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b86255cd-4ef7-474b-a335-a581d64b7cda" containerName="maas-api"
Jul 28 23:00:03.042794 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042537    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="140af177-5044-4bd1-9912-1c7982c4f77e" containerName="maas-api"
Jul 28 23:00:03.042794 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042542    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8cff19b-7cca-4051-a2a6-2ae828c9868d" containerName="maas-api"
Jul 28 23:00:03.042794 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042548    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c474c0ba-3644-4872-9921-4bb7d49856a8" containerName="istio-proxy"
Jul 28 23:00:03.042794 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042554    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e77f7753-2d42-44e2-a0b3-ae63876b18af" containerName="istio-proxy"
Jul 28 23:00:03.042794 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.042560    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e77f7753-2d42-44e2-a0b3-ae63876b18af" containerName="istio-proxy"
Jul 28 23:00:03.045625 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.045607    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s"
Jul 28 23:00:03.047590 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.047568    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lskt7\"/\"kube-root-ca.crt\""
Jul 28 23:00:03.047691 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.047578    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lskt7\"/\"default-dockercfg-6vcgc\""
Jul 28 23:00:03.047774 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.047761    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lskt7\"/\"openshift-service-ca.crt\""
Jul 28 23:00:03.055408 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.055388    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s"]
Jul 28 23:00:03.157634 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.157600    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/689c0c0c-2b1b-417d-a274-f5ccfb4459d6-proc\") pod \"perf-node-gather-daemonset-9hl6s\" (UID: \"689c0c0c-2b1b-417d-a274-f5ccfb4459d6\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s"
Jul 28 23:00:03.157816 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.157645    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/689c0c0c-2b1b-417d-a274-f5ccfb4459d6-sys\") pod \"perf-node-gather-daemonset-9hl6s\" (UID: \"689c0c0c-2b1b-417d-a274-f5ccfb4459d6\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s"
Jul 28 23:00:03.157816 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.157678    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/689c0c0c-2b1b-417d-a274-f5ccfb4459d6-lib-modules\") pod \"perf-node-gather-daemonset-9hl6s\" (UID: \"689c0c0c-2b1b-417d-a274-f5ccfb4459d6\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s"
Jul 28 23:00:03.157816 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.157722    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/689c0c0c-2b1b-417d-a274-f5ccfb4459d6-podres\") pod \"perf-node-gather-daemonset-9hl6s\" (UID: \"689c0c0c-2b1b-417d-a274-f5ccfb4459d6\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s"
Jul 28 23:00:03.157816 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.157756    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnqpj\" (UniqueName: \"kubernetes.io/projected/689c0c0c-2b1b-417d-a274-f5ccfb4459d6-kube-api-access-jnqpj\") pod \"perf-node-gather-daemonset-9hl6s\" (UID: \"689c0c0c-2b1b-417d-a274-f5ccfb4459d6\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s"
Jul 28 23:00:03.258896 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.258863    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/689c0c0c-2b1b-417d-a274-f5ccfb4459d6-sys\") pod \"perf-node-gather-daemonset-9hl6s\" (UID: \"689c0c0c-2b1b-417d-a274-f5ccfb4459d6\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s"
Jul 28 23:00:03.259045 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.258903    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/689c0c0c-2b1b-417d-a274-f5ccfb4459d6-lib-modules\") pod \"perf-node-gather-daemonset-9hl6s\" (UID: \"689c0c0c-2b1b-417d-a274-f5ccfb4459d6\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s"
Jul 28 23:00:03.259045 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.258931    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/689c0c0c-2b1b-417d-a274-f5ccfb4459d6-podres\") pod \"perf-node-gather-daemonset-9hl6s\" (UID: \"689c0c0c-2b1b-417d-a274-f5ccfb4459d6\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s"
Jul 28 23:00:03.259045 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.258948    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnqpj\" (UniqueName: \"kubernetes.io/projected/689c0c0c-2b1b-417d-a274-f5ccfb4459d6-kube-api-access-jnqpj\") pod \"perf-node-gather-daemonset-9hl6s\" (UID: \"689c0c0c-2b1b-417d-a274-f5ccfb4459d6\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s"
Jul 28 23:00:03.259045 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.258979    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/689c0c0c-2b1b-417d-a274-f5ccfb4459d6-proc\") pod \"perf-node-gather-daemonset-9hl6s\" (UID: \"689c0c0c-2b1b-417d-a274-f5ccfb4459d6\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s"
Jul 28 23:00:03.259045 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.258978    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/689c0c0c-2b1b-417d-a274-f5ccfb4459d6-sys\") pod \"perf-node-gather-daemonset-9hl6s\" (UID: \"689c0c0c-2b1b-417d-a274-f5ccfb4459d6\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s"
Jul 28 23:00:03.259045 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.259032    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/689c0c0c-2b1b-417d-a274-f5ccfb4459d6-lib-modules\") pod \"perf-node-gather-daemonset-9hl6s\" (UID: \"689c0c0c-2b1b-417d-a274-f5ccfb4459d6\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s"
Jul 28 23:00:03.259274 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.259039    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/689c0c0c-2b1b-417d-a274-f5ccfb4459d6-proc\") pod \"perf-node-gather-daemonset-9hl6s\" (UID: \"689c0c0c-2b1b-417d-a274-f5ccfb4459d6\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s"
Jul 28 23:00:03.259274 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.259092    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/689c0c0c-2b1b-417d-a274-f5ccfb4459d6-podres\") pod \"perf-node-gather-daemonset-9hl6s\" (UID: \"689c0c0c-2b1b-417d-a274-f5ccfb4459d6\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s"
Jul 28 23:00:03.266354 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.266333    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnqpj\" (UniqueName: \"kubernetes.io/projected/689c0c0c-2b1b-417d-a274-f5ccfb4459d6-kube-api-access-jnqpj\") pod \"perf-node-gather-daemonset-9hl6s\" (UID: \"689c0c0c-2b1b-417d-a274-f5ccfb4459d6\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s"
Jul 28 23:00:03.356026 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.355938    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s"
Jul 28 23:00:03.481445 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:03.481416    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s"]
Jul 28 23:00:03.484071 ip-10-0-143-69 kubenswrapper[2574]: W0728 23:00:03.484042    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod689c0c0c_2b1b_417d_a274_f5ccfb4459d6.slice/crio-5421e09aa91cc035ab4ca98e2c9e5fc2af0db5e6b2610de0a7168bd2087c331d WatchSource:0}: Error finding container 5421e09aa91cc035ab4ca98e2c9e5fc2af0db5e6b2610de0a7168bd2087c331d: Status 404 returned error can't find the container with id 5421e09aa91cc035ab4ca98e2c9e5fc2af0db5e6b2610de0a7168bd2087c331d
Jul 28 23:00:04.268721 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:04.268687    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s" event={"ID":"689c0c0c-2b1b-417d-a274-f5ccfb4459d6","Type":"ContainerStarted","Data":"8e1bdaefa83ca51a0db62006c28229781620e15ce1c9c13e77fa97431fed79db"}
Jul 28 23:00:04.269121 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:04.268741    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s" event={"ID":"689c0c0c-2b1b-417d-a274-f5ccfb4459d6","Type":"ContainerStarted","Data":"5421e09aa91cc035ab4ca98e2c9e5fc2af0db5e6b2610de0a7168bd2087c331d"}
Jul 28 23:00:04.269121 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:04.268768    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s"
Jul 28 23:00:04.283316 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:04.283271    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s" podStartSLOduration=1.283255541 podStartE2EDuration="1.283255541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:00:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:00:04.282435503 +0000 UTC m=+3333.266578121" watchObservedRunningTime="2026-07-28 23:00:04.283255541 +0000 UTC m=+3333.267398156"
Jul 28 23:00:05.461373 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:05.461348    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hqgs4_e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4/dns/0.log"
Jul 28 23:00:05.484536 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:05.484511    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hqgs4_e1c1dfc7-3f16-4aa3-b2d9-56caa765dcc4/kube-rbac-proxy/0.log"
Jul 28 23:00:05.610423 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:05.610392    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-swmbg_da509f2e-8338-4ece-9b46-ef348ba52ebc/dns-node-resolver/0.log"
Jul 28 23:00:06.139749 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:06.139714    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-74b5c9c64-f7qff_e525072e-f5b3-4c68-800c-6292353b55bd/registry/0.log"
Jul 28 23:00:06.197918 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:06.197891    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gnhk8_0efef71b-3b20-4214-914d-5138f89c14fc/node-ca/0.log"
Jul 28 23:00:07.165552 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:07.165529    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799crd6zz_1262efc1-7845-4349-8425-0609954af5c6/istio-proxy/0.log"
Jul 28 23:00:07.342955 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:07.342924    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cd5d84f76-9pg4h_cc63bade-aba2-4894-97a9-d4fc058fad23/kube-auth-proxy/0.log"
Jul 28 23:00:07.401666 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:07.401637    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-xfw54_a55b9c60-64aa-4a57-82d7-6309caa50034/istio-proxy/0.log"
Jul 28 23:00:07.449735 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:07.449699    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-xfw54_a55b9c60-64aa-4a57-82d7-6309caa50034/istio-proxy/1.log"
Jul 28 23:00:07.488172 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:07.488144    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-m6qn9_1afb94a8-a577-4cab-93f7-e3cab182cd83/payload-pre-processing/0.log"
Jul 28 23:00:08.378675 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:08.378640    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n8ks7_4c33216f-bedb-44c6-b91c-2b84dabc9f77/serve-healthcheck-canary/0.log"
Jul 28 23:00:08.964023 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:08.963993    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lhf6r_b5d9fd68-fbef-4b56-ac86-4b26358778e7/kube-rbac-proxy/0.log"
Jul 28 23:00:08.985143 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:08.985120    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lhf6r_b5d9fd68-fbef-4b56-ac86-4b26358778e7/exporter/0.log"
Jul 28 23:00:09.006697 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:09.006676    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lhf6r_b5d9fd68-fbef-4b56-ac86-4b26358778e7/extractor/0.log"
Jul 28 23:00:10.282962 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:10.282929    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9hl6s"
Jul 28 23:00:11.106457 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:11.106426    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-cp5wh_21574045-310f-4d71-9427-477488ae7bbb/manager/0.log"
Jul 28 23:00:11.490159 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:11.490123    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jvrt4_846482be-f263-409b-a5f2-a290e50d2cb8/manager/0.log"
Jul 28 23:00:11.725578 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:11.725512    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-796f547645-h6r2g_86d33c6f-e796-4e0f-984e-a1703cd034a2/manager/0.log"
Jul 28 23:00:11.777464 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:11.777381    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-5nmvv_f8b44918-780b-4433-8136-5bf279e14dd5/server/0.log"
Jul 28 23:00:11.962479 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:11.962448    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-xpb8v_e0162e33-5b05-4e2e-a9a3-c58e95c401dd/manager/0.log"
Jul 28 23:00:12.004958 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:12.004935    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d45996775-fzq6v_b03a58fd-5c5b-44fd-b6c2-cc9a6a455178/manager/0.log"
Jul 28 23:00:12.032586 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:12.032517    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d45996775-fzq6v_b03a58fd-5c5b-44fd-b6c2-cc9a6a455178/copy-manifests/0.log"
Jul 28 23:00:12.375065 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:12.374983    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799crd6zz_1262efc1-7845-4349-8425-0609954af5c6/istio-proxy/0.log"
Jul 28 23:00:12.526907 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:12.526880    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cd5d84f76-9pg4h_cc63bade-aba2-4894-97a9-d4fc058fad23/kube-auth-proxy/0.log"
Jul 28 23:00:12.565910 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:12.565881    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-xfw54_a55b9c60-64aa-4a57-82d7-6309caa50034/istio-proxy/0.log"
Jul 28 23:00:12.614593 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:12.614565    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-xfw54_a55b9c60-64aa-4a57-82d7-6309caa50034/istio-proxy/1.log"
Jul 28 23:00:12.642158 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:12.642087    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-m6qn9_1afb94a8-a577-4cab-93f7-e3cab182cd83/payload-pre-processing/0.log"
Jul 28 23:00:14.184381 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:14.184346    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799crd6zz_1262efc1-7845-4349-8425-0609954af5c6/istio-proxy/0.log"
Jul 28 23:00:14.332076 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:14.332041    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cd5d84f76-9pg4h_cc63bade-aba2-4894-97a9-d4fc058fad23/kube-auth-proxy/0.log"
Jul 28 23:00:14.367244 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:14.367204    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-xfw54_a55b9c60-64aa-4a57-82d7-6309caa50034/istio-proxy/0.log"
Jul 28 23:00:14.417511 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:14.417433    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-xfw54_a55b9c60-64aa-4a57-82d7-6309caa50034/istio-proxy/1.log"
Jul 28 23:00:14.446435 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:14.446407    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-m6qn9_1afb94a8-a577-4cab-93f7-e3cab182cd83/payload-pre-processing/0.log"
Jul 28 23:00:19.854730 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:19.854696    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5vhbh_c7cde400-9ef6-4b09-9cc5-f1d05d424e6c/kube-multus/0.log"
Jul 28 23:00:20.077243 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:20.077202    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mj477_1d63809c-6d3f-4a78-9c1c-24b2cf6223f4/kube-multus-additional-cni-plugins/0.log"
Jul 28 23:00:20.099180 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:20.099157    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mj477_1d63809c-6d3f-4a78-9c1c-24b2cf6223f4/egress-router-binary-copy/0.log"
Jul 28 23:00:20.121681 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:20.121607    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mj477_1d63809c-6d3f-4a78-9c1c-24b2cf6223f4/cni-plugins/0.log"
Jul 28 23:00:20.143781 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:20.143754    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mj477_1d63809c-6d3f-4a78-9c1c-24b2cf6223f4/bond-cni-plugin/0.log"
Jul 28 23:00:20.163954 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:20.163910    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mj477_1d63809c-6d3f-4a78-9c1c-24b2cf6223f4/routeoverride-cni/0.log"
Jul 28 23:00:20.184650 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:20.184625    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mj477_1d63809c-6d3f-4a78-9c1c-24b2cf6223f4/whereabouts-cni-bincopy/0.log"
Jul 28 23:00:20.204733 ip-10-0-143-69 kubenswrapper[2574]: I0728 23:00:20.204709    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mj477_1d63809c-6d3f-4a78-9c1c-24b2cf6223f4/whereabouts-cni/0.log"