Jul 20 19:54:23.650243 ip-10-0-143-191 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 20 19:54:23.650257 ip-10-0-143-191 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 20 19:54:23.650266 ip-10-0-143-191 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 20 19:54:23.650561 ip-10-0-143-191 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 20 19:54:33.812320 ip-10-0-143-191 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 20 19:54:33.812339 ip-10-0-143-191 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 3388a30728d046f0942e3d6824c7d191 --
Jul 20 19:56:23.305230 ip-10-0-143-191 systemd[1]: Starting Kubernetes Kubelet...
Jul 20 19:56:23.609575 ip-10-0-143-191 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 19:56:23.609575 ip-10-0-143-191 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 20 19:56:23.609575 ip-10-0-143-191 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 19:56:23.609575 ip-10-0-143-191 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 20 19:56:23.609575 ip-10-0-143-191 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 19:56:23.611447 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.611358    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 20 19:56:23.616437 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616421    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 19:56:23.616437 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616437    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 19:56:23.616515 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616441    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 19:56:23.616515 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616444    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 19:56:23.616515 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616447    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 19:56:23.616515 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616453    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 19:56:23.616515 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616457    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 19:56:23.616515 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616460    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 19:56:23.616515 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616464    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 19:56:23.616515 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616467    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 19:56:23.616515 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616470    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 19:56:23.616515 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616473    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 19:56:23.616515 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616475    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 19:56:23.616515 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616478    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 19:56:23.616515 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616481    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 19:56:23.616515 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616484    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 19:56:23.616515 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616486    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 19:56:23.616515 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616489    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 19:56:23.616515 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616492    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 19:56:23.616515 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616500    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 19:56:23.616515 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616503    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 19:56:23.616984 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616506    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 19:56:23.616984 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616508    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 19:56:23.616984 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616511    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 19:56:23.616984 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616514    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 19:56:23.616984 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616516    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 19:56:23.616984 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616519    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 19:56:23.616984 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616522    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 19:56:23.616984 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616525    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 19:56:23.616984 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616528    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 19:56:23.616984 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616530    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 19:56:23.616984 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616533    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 19:56:23.616984 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616536    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 19:56:23.616984 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616540    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 19:56:23.616984 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616543    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 19:56:23.616984 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616547    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 19:56:23.616984 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616549    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 19:56:23.616984 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616552    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 19:56:23.616984 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616554    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 19:56:23.616984 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616557    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 19:56:23.616984 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616559    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 19:56:23.617470 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616563    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 19:56:23.617470 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616565    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 19:56:23.617470 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616568    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 19:56:23.617470 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616570    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 19:56:23.617470 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616574    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 19:56:23.617470 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616578    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 19:56:23.617470 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616581    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 19:56:23.617470 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616584    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 19:56:23.617470 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616587    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 19:56:23.617470 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616590    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 19:56:23.617470 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616593    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 19:56:23.617470 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616595    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 19:56:23.617470 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616598    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 19:56:23.617470 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616601    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 19:56:23.617470 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616604    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 19:56:23.617470 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616607    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 19:56:23.617470 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616609    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 19:56:23.617470 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616612    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 19:56:23.617470 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616615    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 19:56:23.617959 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616617    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 19:56:23.617959 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616620    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 19:56:23.617959 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616623    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 19:56:23.617959 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616625    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 19:56:23.617959 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616628    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 19:56:23.617959 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616630    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 19:56:23.617959 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616633    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 19:56:23.617959 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616637    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 19:56:23.617959 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616640    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 19:56:23.617959 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616642    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 19:56:23.617959 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616644    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 19:56:23.617959 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616648    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 19:56:23.617959 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616650    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 19:56:23.617959 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616653    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 19:56:23.617959 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616656    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 19:56:23.617959 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616659    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 19:56:23.617959 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616661    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 19:56:23.617959 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616664    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 19:56:23.617959 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616667    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 19:56:23.618408 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616670    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 19:56:23.618408 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616672    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 19:56:23.618408 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616675    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 19:56:23.618408 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616677    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 19:56:23.618408 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616680    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 19:56:23.618408 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616682    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 19:56:23.618408 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616685    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 19:56:23.618408 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616688    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 19:56:23.618408 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616691    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 19:56:23.618408 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616694    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 19:56:23.618408 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616697    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 19:56:23.618408 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616700    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 19:56:23.618408 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616702    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 19:56:23.618408 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616705    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 19:56:23.618408 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616707    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 19:56:23.618408 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616709    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 19:56:23.618408 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616712    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 19:56:23.618408 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616715    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 19:56:23.618408 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616717    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 19:56:23.618408 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.616720    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 19:56:23.619259 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619246    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 19:56:23.619259 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619259    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 19:56:23.619322 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619263    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 19:56:23.619322 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619266    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 19:56:23.619322 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619269    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 19:56:23.619322 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619273    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 19:56:23.619322 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619276    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 19:56:23.619322 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619279    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 19:56:23.619322 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619281    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 19:56:23.619322 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619284    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 19:56:23.619322 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619287    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 19:56:23.619322 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619289    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 19:56:23.619322 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619292    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 19:56:23.619322 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619295    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 19:56:23.619322 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619297    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 19:56:23.619322 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619300    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 19:56:23.619322 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619302    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 19:56:23.619322 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619305    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 19:56:23.619322 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619307    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 19:56:23.619322 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619310    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 19:56:23.619322 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619312    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 19:56:23.619322 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619315    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 19:56:23.619811 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619317    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 19:56:23.619811 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619320    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 19:56:23.619811 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619322    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 19:56:23.619811 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619326    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 19:56:23.619811 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619330    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 19:56:23.619811 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619333    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 19:56:23.619811 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619336    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 19:56:23.619811 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619338    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 19:56:23.619811 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619341    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 19:56:23.619811 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619344    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 19:56:23.619811 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619347    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 19:56:23.619811 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619349    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 19:56:23.619811 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619352    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 19:56:23.619811 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619354    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 19:56:23.619811 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619357    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 19:56:23.619811 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619359    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 19:56:23.619811 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619362    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 19:56:23.619811 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619365    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 19:56:23.619811 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619367    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 19:56:23.620293 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619370    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 19:56:23.620293 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619372    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 19:56:23.620293 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619374    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 19:56:23.620293 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619378    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 19:56:23.620293 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619380    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 19:56:23.620293 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619383    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 19:56:23.620293 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619385    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 19:56:23.620293 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619388    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 19:56:23.620293 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619390    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 19:56:23.620293 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619394    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 19:56:23.620293 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619398    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 19:56:23.620293 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619401    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 19:56:23.620293 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619405    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 19:56:23.620293 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619408    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 19:56:23.620293 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619411    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 19:56:23.620293 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619415    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 19:56:23.620293 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619418    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 19:56:23.620293 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619420    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 19:56:23.620293 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619423    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 19:56:23.620293 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619425    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 19:56:23.620784 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619428    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 19:56:23.620784 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619430    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 19:56:23.620784 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619434    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 19:56:23.620784 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619436    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 19:56:23.620784 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619438    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 19:56:23.620784 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619441    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 19:56:23.620784 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619443    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 19:56:23.620784 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619446    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 19:56:23.620784 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619449    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 19:56:23.620784 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619451    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 19:56:23.620784 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619455    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 19:56:23.620784 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619457    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 19:56:23.620784 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619460    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 19:56:23.620784 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619462    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 19:56:23.620784 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619464    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 19:56:23.620784 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619467    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 19:56:23.620784 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619469    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 19:56:23.620784 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619472    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 19:56:23.620784 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619475    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 19:56:23.620784 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619477    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 19:56:23.621303 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619480    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 19:56:23.621303 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619482    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 19:56:23.621303 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619485    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 19:56:23.621303 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619487    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 19:56:23.621303 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619490    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 19:56:23.621303 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619493    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 19:56:23.621303 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619496    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 19:56:23.621303 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619499    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 19:56:23.621303 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619502    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 19:56:23.621303 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619504    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 19:56:23.621303 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619508    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 19:56:23.621303 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619512    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 19:56:23.621303 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619514    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 19:56:23.621303 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619517    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 19:56:23.621303 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619520    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 19:56:23.621303 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619523    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 19:56:23.621303 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619525    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 19:56:23.621303 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619528    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 19:56:23.621303 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619947    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 19:56:23.621303 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619953    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 19:56:23.621777 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619956    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 19:56:23.621777 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619959    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 19:56:23.621777 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619963    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 19:56:23.621777 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619966    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 19:56:23.621777 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619968    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 19:56:23.621777 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619971    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 19:56:23.621777 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619974    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 19:56:23.621777 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619976    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 19:56:23.621777 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619979    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 19:56:23.621777 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619982    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 19:56:23.621777 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619985    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 19:56:23.621777 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619987    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 19:56:23.621777 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619990    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 19:56:23.621777 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619992    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 19:56:23.621777 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619995    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 19:56:23.621777 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.619997    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 19:56:23.621777 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620000    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 19:56:23.621777 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620002    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 19:56:23.621777 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620005    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 19:56:23.622264 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620010    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 19:56:23.622264 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620013    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 19:56:23.622264 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620016    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 19:56:23.622264 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620019    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 19:56:23.622264 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620022    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 19:56:23.622264 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620026    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 19:56:23.622264 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620030    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 19:56:23.622264 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620033    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 19:56:23.622264 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620036    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 19:56:23.622264 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620038    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 19:56:23.622264 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620041    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 19:56:23.622264 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620044    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 19:56:23.622264 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620046    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 19:56:23.622264 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620048    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 19:56:23.622264 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620051    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 19:56:23.622264 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620053    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 19:56:23.622264 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620056    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 19:56:23.622264 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620059    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 19:56:23.622264 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620061    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 19:56:23.622264 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620064    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 19:56:23.622774 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620066    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 19:56:23.622774 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620069    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 19:56:23.622774 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620072    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 19:56:23.622774 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620075    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 19:56:23.622774 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620078    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 19:56:23.622774 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620080    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 19:56:23.622774 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620083    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 19:56:23.622774 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620085    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 19:56:23.622774 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620088    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 19:56:23.622774 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620090    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 19:56:23.622774 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620092    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 19:56:23.622774 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620097    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 19:56:23.622774 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620100    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 19:56:23.622774 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620103    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 19:56:23.622774 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620106    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 19:56:23.622774 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620109    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 19:56:23.622774 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620112    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 19:56:23.622774 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620115    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 19:56:23.622774 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620118    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 19:56:23.623247 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620120    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 19:56:23.623247 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620122    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 19:56:23.623247 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620125    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 19:56:23.623247 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620127    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 19:56:23.623247 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620130    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 19:56:23.623247 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620132    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 19:56:23.623247 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620135    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 19:56:23.623247 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620137    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 19:56:23.623247 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620140    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 19:56:23.623247 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620142    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 19:56:23.623247 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620145    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 19:56:23.623247 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620148    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 19:56:23.623247 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620150    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 19:56:23.623247 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620153    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 19:56:23.623247 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620156    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 19:56:23.623247 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620158    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 19:56:23.623247 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620161    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 19:56:23.623247 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620163    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 19:56:23.623247 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620166    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 19:56:23.623247 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620168    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 19:56:23.623717 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620171    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 19:56:23.623717 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620173    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 19:56:23.623717 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620176    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 19:56:23.623717 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620179    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 19:56:23.623717 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620183    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 19:56:23.623717 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620185    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 19:56:23.623717 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620188    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 19:56:23.623717 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620190    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 19:56:23.623717 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620193    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 19:56:23.623717 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620195    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 19:56:23.623717 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620198    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 19:56:23.623717 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620201    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 19:56:23.623717 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620203    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 19:56:23.623717 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620206    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 19:56:23.623717 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620208    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 19:56:23.623717 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620211    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 19:56:23.623717 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620213    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 19:56:23.623717 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620216    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 19:56:23.623717 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620218    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 19:56:23.623717 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620284    2580 flags.go:64] FLAG: --address="0.0.0.0"
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620291    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620297    2580 flags.go:64] FLAG: --anonymous-auth="true"
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620301    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620306    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620309    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620314    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620318    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620321    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620324    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620327    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620331    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620334    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620337    2580 flags.go:64] FLAG: --cgroup-root=""
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620340    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620343    2580 flags.go:64] FLAG: --client-ca-file=""
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620346    2580 flags.go:64] FLAG: --cloud-provider="external"
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620349    2580 flags.go:64] FLAG: --cluster-dns="[]"
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620353    2580 flags.go:64] FLAG: --cluster-domain=""
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620356    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620359    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620362    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620366    2580 flags.go:64] FLAG: --container-log-max-files="5"
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620369    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 20 19:56:23.624247 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620372    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620375    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620378    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620381    2580 flags.go:64] FLAG: --contention-profiling="false"
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620384    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620387    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620390    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620393    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620397    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620400    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620403    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620405    2580 flags.go:64] FLAG: --enable-load-reader="false"
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620408    2580 flags.go:64] FLAG: --enable-server="true"
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620412    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620416    2580 flags.go:64] FLAG: --event-burst="100"
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620419    2580 flags.go:64] FLAG: --event-qps="50"
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620422    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620425    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620428    2580 flags.go:64] FLAG: --eviction-hard=""
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620432    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620434    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620437    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620440    2580 flags.go:64] FLAG: --eviction-soft=""
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620443    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620449    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 20 19:56:23.624834 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620452    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620455    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620457    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620460    2580 flags.go:64] FLAG: --fail-swap-on="true"
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620463    2580 flags.go:64] FLAG: --feature-gates=""
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620467    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620470    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620473    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620476    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620479    2580 flags.go:64] FLAG: --healthz-port="10248"
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620482    2580 flags.go:64] FLAG: --help="false"
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620485    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-143-191.ec2.internal"
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620487    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620490    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620494    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620497    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620500    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620503    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620506    2580 flags.go:64] FLAG: --image-service-endpoint=""
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620509    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620512    2580 flags.go:64] FLAG: --kube-api-burst="100"
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620515    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620519    2580 flags.go:64] FLAG: --kube-api-qps="50"
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620521    2580 flags.go:64] FLAG: --kube-reserved=""
Jul 20 19:56:23.625559 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620524    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620527    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620530    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620533    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620535    2580 flags.go:64] FLAG: --lock-file=""
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620538    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620541    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620545    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620550    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620553    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620556    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620559    2580 flags.go:64] FLAG: --logging-format="text"
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620562    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620565    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620568    2580 flags.go:64] FLAG: --manifest-url=""
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620571    2580 flags.go:64] FLAG: --manifest-url-header=""
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620575    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620578    2580 flags.go:64] FLAG: --max-open-files="1000000"
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620582    2580 flags.go:64] FLAG: --max-pods="110"
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620585    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620588    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620591    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620594    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620597    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620600    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 20 19:56:23.626180 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620603    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620609    2580 flags.go:64] FLAG: --node-status-max-images="50"
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620612    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620615    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620618    2580 flags.go:64] FLAG: --pod-cidr=""
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620622    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620627    2580 flags.go:64] FLAG: --pod-manifest-path=""
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620630    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620633    2580 flags.go:64] FLAG: --pods-per-core="0"
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620636    2580 flags.go:64] FLAG: --port="10250"
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620639    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620642    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0eea04f7a0c139803"
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620645    2580 flags.go:64] FLAG: --qos-reserved=""
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620647    2580 flags.go:64] FLAG: --read-only-port="10255"
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620650    2580 flags.go:64] FLAG: --register-node="true"
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620655    2580 flags.go:64] FLAG: --register-with-taints=""
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620659    2580 flags.go:64] FLAG: --registry-burst="10"
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620662    2580 flags.go:64] FLAG: --registry-qps="5"
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620664    2580 flags.go:64] FLAG: --reserved-cpus=""
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620667    2580 flags.go:64] FLAG: --reserved-memory=""
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620671    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620673    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620677    2580 flags.go:64] FLAG: --rotate-certificates="false"
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620679    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620682    2580 flags.go:64] FLAG: --runonce="false"
Jul 20 19:56:23.626821 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620685    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620688    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620691    2580 flags.go:64] FLAG: --seccomp-default="false"
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620694    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620696    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620699    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620703    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620706    2580 flags.go:64] FLAG: --storage-driver-password="root"
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620708    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620711    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620714    2580 flags.go:64] FLAG: --storage-driver-user="root"
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620716    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620719    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620723    2580 flags.go:64] FLAG: --system-cgroups=""
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620726    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620731    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620733    2580 flags.go:64] FLAG: --tls-cert-file=""
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620736    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620742    2580 flags.go:64] FLAG: --tls-min-version=""
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620745    2580 flags.go:64] FLAG: --tls-private-key-file=""
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620747    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620750    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620754    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620758    2580 flags.go:64] FLAG: --v="2"
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620761    2580 flags.go:64] FLAG: --version="false"
Jul 20 19:56:23.627446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620765    2580 flags.go:64] FLAG: --vmodule=""
Jul 20 19:56:23.628100 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620768    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 20 19:56:23.628100 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.620772    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 20 19:56:23.628100 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620889    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 19:56:23.628100 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620892    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 19:56:23.628100 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620896    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 19:56:23.628100 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620899    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 19:56:23.628100 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620901    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 19:56:23.628100 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620904    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 19:56:23.628100 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620907    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 19:56:23.628100 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620909    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 19:56:23.628100 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620929    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 19:56:23.628100 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620935    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 19:56:23.628100 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620939    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 19:56:23.628100 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620942    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 19:56:23.628100 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620945    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 19:56:23.628100 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620948    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 19:56:23.628100 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620950    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 19:56:23.628100 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620953    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 19:56:23.628100 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620956    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 19:56:23.628564 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620958    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 19:56:23.628564 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620962    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 19:56:23.628564 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620965    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 19:56:23.628564 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620967    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 19:56:23.628564 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620970    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 19:56:23.628564 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620974    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 19:56:23.628564 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620976    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 19:56:23.628564 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620979    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 19:56:23.628564 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620982    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 19:56:23.628564 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620986    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 19:56:23.628564 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620989    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 19:56:23.628564 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620991    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 19:56:23.628564 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620994    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 19:56:23.628564 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620996    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 19:56:23.628564 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.620999    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 19:56:23.628564 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621002    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 19:56:23.628564 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621004    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 19:56:23.628564 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621007    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 19:56:23.628564 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621010    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 19:56:23.628564 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621012    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 19:56:23.629069 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621015    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 19:56:23.629069 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621017    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 19:56:23.629069 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621020    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 19:56:23.629069 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621022    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 19:56:23.629069 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621025    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 19:56:23.629069 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621028    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 19:56:23.629069 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621030    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 19:56:23.629069 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621033    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 19:56:23.629069 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621036    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 19:56:23.629069 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621039    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 19:56:23.629069 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621043    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 19:56:23.629069 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621045    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 19:56:23.629069 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621048    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 19:56:23.629069 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621051    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 19:56:23.629069 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621055    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 19:56:23.629069 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621058    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 19:56:23.629069 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621060    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 19:56:23.629069 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621064    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 19:56:23.629069 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621066    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 19:56:23.629069 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621069    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 19:56:23.629554 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621072    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 19:56:23.629554 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621076    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 19:56:23.629554 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621078    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 19:56:23.629554 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621081    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 19:56:23.629554 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621084    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 19:56:23.629554 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621086    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 19:56:23.629554 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621089    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 19:56:23.629554 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621092    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 19:56:23.629554 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621094    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 19:56:23.629554 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621096    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 19:56:23.629554 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621099    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 19:56:23.629554 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621102    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 19:56:23.629554 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621105    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 19:56:23.629554 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621108    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 19:56:23.629554 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621110    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 19:56:23.629554 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621113    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 19:56:23.629554 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621115    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 19:56:23.629554 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621118    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 19:56:23.629554 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621120    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 19:56:23.629554 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621123    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 19:56:23.630053 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621125    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 19:56:23.630053 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621128    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 19:56:23.630053 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621131    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 19:56:23.630053 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621133    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 19:56:23.630053 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621137    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 19:56:23.630053 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621140    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 19:56:23.630053 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621143    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 19:56:23.630053 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621146    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 19:56:23.630053 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621149    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 19:56:23.630053 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621153    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 19:56:23.630053 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621155    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 19:56:23.630053 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621158    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 19:56:23.630053 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621160    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 19:56:23.630053 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621164    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 19:56:23.630053 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621166    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 19:56:23.630053 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621169    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 19:56:23.630053 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621171    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 19:56:23.630053 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621174    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 19:56:23.630053 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621177    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 19:56:23.630511 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621179    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 19:56:23.630511 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621182    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 19:56:23.630511 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.621184    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 19:56:23.630511 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.621661    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 19:56:23.630511 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.628719    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 20 19:56:23.630511 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.628736    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 20 19:56:23.630511 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.628766    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 20 19:56:23.630511 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.628774    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 20 19:56:23.630511 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628850    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 19:56:23.630511 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628857    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 19:56:23.630511 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628861    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 19:56:23.630511 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628866    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 19:56:23.630511 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628870    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 19:56:23.630511 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628874    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 19:56:23.630860 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628878    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 19:56:23.630860 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628881    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 19:56:23.630860 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628886    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 19:56:23.630860 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628890    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 19:56:23.630860 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628894    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 19:56:23.630860 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628898    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 19:56:23.630860 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628903    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 19:56:23.630860 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628907    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 19:56:23.630860 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628926    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 19:56:23.630860 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628931    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 19:56:23.630860 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628935    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 19:56:23.630860 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628938    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 19:56:23.630860 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628942    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 19:56:23.630860 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628946    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 19:56:23.630860 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628950    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 19:56:23.630860 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628954    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 19:56:23.630860 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628958    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 19:56:23.630860 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628963    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 19:56:23.630860 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628967    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 19:56:23.630860 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628971    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 19:56:23.631393 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628974    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 19:56:23.631393 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628978    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 19:56:23.631393 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628983    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 19:56:23.631393 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628987    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 19:56:23.631393 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628992    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 19:56:23.631393 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628996    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 19:56:23.631393 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.628999    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 19:56:23.631393 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629003    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 19:56:23.631393 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629007    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 19:56:23.631393 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629010    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 19:56:23.631393 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629014    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 19:56:23.631393 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629018    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 19:56:23.631393 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629022    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 19:56:23.631393 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629025    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 19:56:23.631393 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629029    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 19:56:23.631393 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629033    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 19:56:23.631393 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629037    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 19:56:23.631393 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629040    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 19:56:23.631393 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629044    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 19:56:23.631393 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629048    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 19:56:23.631887 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629052    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 19:56:23.631887 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629056    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 19:56:23.631887 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629059    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 19:56:23.631887 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629063    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 19:56:23.631887 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629067    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 19:56:23.631887 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629071    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 19:56:23.631887 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629074    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 19:56:23.631887 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629079    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 19:56:23.631887 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629083    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 19:56:23.631887 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629087    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 19:56:23.631887 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629091    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 19:56:23.631887 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629094    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 19:56:23.631887 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629098    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 19:56:23.631887 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629103    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 19:56:23.631887 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629107    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 19:56:23.631887 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629111    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 19:56:23.631887 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629115    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 19:56:23.631887 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629119    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 19:56:23.631887 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629122    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 19:56:23.632352 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629125    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 19:56:23.632352 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629129    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 19:56:23.632352 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629133    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 19:56:23.632352 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629137    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 19:56:23.632352 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629143    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 19:56:23.632352 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629148    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 19:56:23.632352 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629153    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 19:56:23.632352 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629157    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 19:56:23.632352 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629161    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 19:56:23.632352 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629165    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 19:56:23.632352 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629171    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 19:56:23.632352 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629176    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 19:56:23.632352 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629180    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 19:56:23.632352 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629186    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 19:56:23.632352 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629191    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 19:56:23.632352 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629194    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 19:56:23.632352 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629198    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 19:56:23.632352 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629202    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 19:56:23.632352 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629206    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 19:56:23.632352 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629209    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 19:56:23.632869 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629212    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 19:56:23.632869 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629216    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 19:56:23.632869 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629220    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 19:56:23.632869 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629224    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 19:56:23.632869 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629227    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 19:56:23.632869 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629231    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 19:56:23.632869 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629235    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 19:56:23.632869 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629239    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 19:56:23.632869 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629243    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 19:56:23.632869 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629248    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 19:56:23.632869 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629254    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 19:56:23.632869 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629258    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 19:56:23.632869 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629262    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 19:56:23.632869 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629266    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 19:56:23.632869 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.629274    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 19:56:23.633265 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629432    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 19:56:23.633265 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629439    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 19:56:23.633265 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629443    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 19:56:23.633265 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629448    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 19:56:23.633265 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629452    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 19:56:23.633265 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629457    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 19:56:23.633265 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629461    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 19:56:23.633265 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629464    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 19:56:23.633265 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629468    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 19:56:23.633265 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629472    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 19:56:23.633265 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629476    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 19:56:23.633265 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629481    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 19:56:23.633265 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629485    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 19:56:23.633265 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629488    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 19:56:23.633265 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629492    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 19:56:23.633265 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629496    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 19:56:23.633265 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629500    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 19:56:23.633265 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629504    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 19:56:23.633265 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629508    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 19:56:23.633714 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629512    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 19:56:23.633714 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629515    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 19:56:23.633714 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629519    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 19:56:23.633714 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629526    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 19:56:23.633714 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629531    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 19:56:23.633714 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629534    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 19:56:23.633714 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629539    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 19:56:23.633714 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629543    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 19:56:23.633714 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629547    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 19:56:23.633714 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629550    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 19:56:23.633714 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629554    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 19:56:23.633714 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629558    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 19:56:23.633714 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629562    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 19:56:23.633714 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629566    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 19:56:23.633714 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629569    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 19:56:23.633714 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629574    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 19:56:23.633714 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629577    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 19:56:23.633714 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629581    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 19:56:23.633714 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629584    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 19:56:23.634213 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629588    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 19:56:23.634213 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629592    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 19:56:23.634213 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629596    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 19:56:23.634213 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629600    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 19:56:23.634213 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629603    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 19:56:23.634213 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629607    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 19:56:23.634213 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629611    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 19:56:23.634213 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629615    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 19:56:23.634213 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629619    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 19:56:23.634213 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629624    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 19:56:23.634213 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629629    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 19:56:23.634213 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629633    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 19:56:23.634213 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629636    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 19:56:23.634213 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629640    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 19:56:23.634213 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629644    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 19:56:23.634213 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629648    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 19:56:23.634213 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629651    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 19:56:23.634213 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629655    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 19:56:23.634213 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629659    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 19:56:23.634213 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629663    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 19:56:23.634679 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629667    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 19:56:23.634679 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629670    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 19:56:23.634679 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629673    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 19:56:23.634679 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629677    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 19:56:23.634679 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629682    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 19:56:23.634679 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629686    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 19:56:23.634679 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629689    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 19:56:23.634679 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629693    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 19:56:23.634679 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629697    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 19:56:23.634679 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629700    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 19:56:23.634679 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629704    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 19:56:23.634679 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629708    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 19:56:23.634679 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629712    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 19:56:23.634679 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629716    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 19:56:23.634679 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629719    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 19:56:23.634679 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629723    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 19:56:23.634679 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629727    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 19:56:23.634679 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629732    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 19:56:23.634679 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629737    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 19:56:23.635256 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629742    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 19:56:23.635256 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629746    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 19:56:23.635256 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629751    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 19:56:23.635256 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629755    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 19:56:23.635256 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629759    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 19:56:23.635256 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629763    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 19:56:23.635256 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629767    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 19:56:23.635256 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629771    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 19:56:23.635256 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629774    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 19:56:23.635256 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629778    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 19:56:23.635256 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629782    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 19:56:23.635256 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629785    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 19:56:23.635256 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629789    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 19:56:23.635256 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629793    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 19:56:23.635256 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629796    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 19:56:23.635256 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629800    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 19:56:23.635256 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629804    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 19:56:23.635256 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629815    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 19:56:23.635256 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629820    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 19:56:23.635256 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629824    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 19:56:23.635256 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629828    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 19:56:23.635761 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:23.629831    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 19:56:23.635761 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.629838    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 19:56:23.635761 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.629997    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 20 19:56:23.635761 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.632423    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 20 19:56:23.635761 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.633696    2580 server.go:1019] "Starting client certificate rotation"
Jul 20 19:56:23.635761 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.633791    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 20 19:56:23.635761 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.633858    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 20 19:56:23.636015 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.636001    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 20 19:56:23.640480 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.640458    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 20 19:56:23.647266 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.647250    2580 log.go:25] "Validated CRI v1 runtime API"
Jul 20 19:56:23.649968 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.649951    2580 log.go:25] "Validated CRI v1 image API"
Jul 20 19:56:23.651377 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.651359    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 20 19:56:23.653387 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.653368    2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8ae45e33-ce7d-4d84-97f2-63561939e7c3:/dev/nvme0n1p4 fbbdd5bd-0f58-465b-8bd7-47e63f80d134:/dev/nvme0n1p3]
Jul 20 19:56:23.653436 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.653390    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 20 19:56:23.658529 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.658419    2580 manager.go:216] Machine: {Timestamp:2026-07-20 19:56:23.657224543 +0000 UTC m=+0.260339018 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096456 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d986cbc71c4caae7245f38e2d44a3 SystemUUID:ec2d986c-bc71-c4ca-ae72-45f38e2d44a3 BootID:3388a307-28d0-46f0-942e-3d6824c7d191 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6c:c3:5e:3e:f1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6c:c3:5e:3e:f1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:b5:25:57:7a:2e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 20 19:56:23.658529 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.658526    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 20 19:56:23.658639 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.658606    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 20 19:56:23.660662 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.660638    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 20 19:56:23.660807 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.660665    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-191.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 20 19:56:23.661758 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.661748    2580 topology_manager.go:138] "Creating topology manager with none policy"
Jul 20 19:56:23.661791 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.661761    2580 container_manager_linux.go:309] "Creating device plugin manager"
Jul 20 19:56:23.661791 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.661768    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 20 19:56:23.661791 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.661778    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 20 19:56:23.662958 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.662948    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 20 19:56:23.666672 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.666661    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 20 19:56:23.666803 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.666793    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 20 19:56:23.667507 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.667497    2580 kubelet.go:486] "Attempting to sync node with API server"
Jul 20 19:56:23.667547 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.667516    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 20 19:56:23.667547 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.667537    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 20 19:56:23.667617 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.667552    2580 kubelet.go:398] "Adding apiserver pod source"
Jul 20 19:56:23.667617 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.667566    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 20 19:56:23.667962 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.667951    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 20 19:56:23.667997 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.667970    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 20 19:56:23.669316 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.669299    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 20 19:56:23.669722 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.669697    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 20 19:56:23.671406 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.671393    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 20 19:56:23.671456 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.671423    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 20 19:56:23.672338 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.672326    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 20 19:56:23.672390 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.672342    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 20 19:56:23.672390 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.672348    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 20 19:56:23.672390 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.672354    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 20 19:56:23.672390 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.672363    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 20 19:56:23.672390 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.672369    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 20 19:56:23.672390 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.672378    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 20 19:56:23.672390 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.672383    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 20 19:56:23.672390 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.672390    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 20 19:56:23.672601 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.672396    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 20 19:56:23.672601 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.672405    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 20 19:56:23.672601 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.672410    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 20 19:56:23.673127 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.673101    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 20 19:56:23.673127 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.673114    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 20 19:56:23.676233 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.676217    2580 server.go:1268] "Started kubelet"
Jul 20 19:56:23.676403 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.676373    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 20 19:56:23.676497 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.676419    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 20 19:56:23.676865 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.676667    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 20 19:56:23.679304 ip-10-0-143-191 systemd[1]: Started Kubernetes Kubelet.
Jul 20 19:56:23.679645 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.679599    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 20 19:56:23.680611 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:23.680583    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 20 19:56:23.681300 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:23.681277    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-191.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 20 19:56:23.681524 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.681304    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-191.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 20 19:56:23.683557 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.683541    2580 server.go:310] "Adding debug handlers to kubelet server"
Jul 20 19:56:23.683697 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:23.683673    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 20 19:56:23.686388 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.686369    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 20 19:56:23.686388 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.686370    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 20 19:56:23.686531 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:23.685674    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-191.ec2.internal.18c417ace4371753  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-191.ec2.internal,UID:ip-10-0-143-191.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-191.ec2.internal,},FirstTimestamp:2026-07-20 19:56:23.676188499 +0000 UTC m=+0.279302975,LastTimestamp:2026-07-20 19:56:23.676188499 +0000 UTC m=+0.279302975,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-191.ec2.internal,}"
Jul 20 19:56:23.686531 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:23.686518    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-191.ec2.internal\" not found"
Jul 20 19:56:23.687326 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.687253    2580 factory.go:55] Registering systemd factory
Jul 20 19:56:23.687326 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.687271    2580 factory.go:223] Registration of the systemd container factory successfully
Jul 20 19:56:23.687326 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.687278    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 20 19:56:23.687497 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.687335    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 20 19:56:23.687497 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.687347    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 20 19:56:23.687497 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.687473    2580 factory.go:153] Registering CRI-O factory
Jul 20 19:56:23.687497 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.687488    2580 factory.go:223] Registration of the crio container factory successfully
Jul 20 19:56:23.687725 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.687535    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 20 19:56:23.687725 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.687557    2580 factory.go:103] Registering Raw factory
Jul 20 19:56:23.687725 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.687573    2580 manager.go:1196] Started watching for new ooms in manager
Jul 20 19:56:23.687989 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.687976    2580 manager.go:349] Starting recovery of all containers
Jul 20 19:56:23.689436 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.689420    2580 reconstruct.go:97] "Volume reconstruction finished"
Jul 20 19:56:23.689436 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.689436    2580 reconciler.go:29] "Reconciler: start to sync state"
Jul 20 19:56:23.691336 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.691157    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bnwgz"
Jul 20 19:56:23.695548 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:23.695515    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 20 19:56:23.695651 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:23.695605    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-191.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 20 19:56:23.697082 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.697061    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bnwgz"
Jul 20 19:56:23.699734 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.699718    2580 manager.go:354] Recovery completed
Jul 20 19:56:23.704107 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.704095    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 19:56:23.706293 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.706271    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-191.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 19:56:23.706383 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.706306    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-191.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 19:56:23.706383 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.706320    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-191.ec2.internal" event="NodeHasSufficientPID"
Jul 20 19:56:23.706863 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.706848    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 20 19:56:23.706863 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.706860    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 20 19:56:23.706992 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.706874    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 20 19:56:23.706992 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.706939    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 19:56:23.707637 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.707622    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-191.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 19:56:23.707702 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.707647    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-191.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 19:56:23.707702 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.707657    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-191.ec2.internal" event="NodeHasSufficientPID"
Jul 20 19:56:23.708246 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:23.708166    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-191.ec2.internal.18c417ace6026f2f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-191.ec2.internal,UID:ip-10-0-143-191.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-191.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-191.ec2.internal,},FirstTimestamp:2026-07-20 19:56:23.706292015 +0000 UTC m=+0.309406497,LastTimestamp:2026-07-20 19:56:23.706292015 +0000 UTC m=+0.309406497,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-191.ec2.internal,}"
Jul 20 19:56:23.708893 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.708882    2580 policy_none.go:49] "None policy: Start"
Jul 20 19:56:23.709023 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.708897    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 20 19:56:23.709023 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.708907    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 20 19:56:23.710858 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.710848    2580 policy_none.go:47] "Start"
Jul 20 19:56:23.745665 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.745484    2580 manager.go:337] "Starting Device Plugin manager"
Jul 20 19:56:23.745758 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:23.745705    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 20 19:56:23.745758 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.745717    2580 server.go:87] "Starting device plugin registration server"
Jul 20 19:56:23.745986 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.745966    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 20 19:56:23.746069 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.745983    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 20 19:56:23.746136 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.746086    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 20 19:56:23.746189 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.746160    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 20 19:56:23.746189 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.746170    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 20 19:56:23.746687 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:23.746665    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 20 19:56:23.746773 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:23.746714    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-191.ec2.internal\" not found"
Jul 20 19:56:23.829318 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.829278    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 20 19:56:23.831489 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.830497    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 20 19:56:23.831489 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.830535    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 20 19:56:23.831489 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.830567    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 20 19:56:23.831489 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:23.830610    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 20 19:56:23.832701 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.832684    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 19:56:23.846640 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.846624    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 19:56:23.847403 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.847385    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-191.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 19:56:23.847483 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.847418    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-191.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 19:56:23.847483 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.847434    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-191.ec2.internal" event="NodeHasSufficientPID"
Jul 20 19:56:23.847483 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.847457    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-191.ec2.internal"
Jul 20 19:56:23.855136 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.855124    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-191.ec2.internal"
Jul 20 19:56:23.855186 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:23.855144    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-191.ec2.internal\": node \"ip-10-0-143-191.ec2.internal\" not found"
Jul 20 19:56:23.871130 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:23.871078    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-191.ec2.internal\" not found"
Jul 20 19:56:23.931455 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.931419    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-191.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-191.ec2.internal"]
Jul 20 19:56:23.931609 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.931518    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 19:56:23.932880 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.932864    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-191.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 19:56:23.932955 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.932899    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-191.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 19:56:23.932955 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.932909    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-191.ec2.internal" event="NodeHasSufficientPID"
Jul 20 19:56:23.935261 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.935249    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 19:56:23.935411 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.935396    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-191.ec2.internal"
Jul 20 19:56:23.935452 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.935427    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 19:56:23.935966 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.935950    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-191.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 19:56:23.936046 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.935969    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-191.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 19:56:23.936046 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.935977    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-191.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 19:56:23.936046 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.935990    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-191.ec2.internal" event="NodeHasSufficientPID"
Jul 20 19:56:23.936046 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.935992    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-191.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 19:56:23.936046 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.936002    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-191.ec2.internal" event="NodeHasSufficientPID"
Jul 20 19:56:23.938235 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.938222    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-191.ec2.internal"
Jul 20 19:56:23.938277 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.938248    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 19:56:23.938946 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.938932    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-191.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 19:56:23.939044 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.938957    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-191.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 19:56:23.939044 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.938971    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-191.ec2.internal" event="NodeHasSufficientPID"
Jul 20 19:56:23.953532 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:23.953507    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-191.ec2.internal\" not found" node="ip-10-0-143-191.ec2.internal"
Jul 20 19:56:23.957341 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:23.957325    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-191.ec2.internal\" not found" node="ip-10-0-143-191.ec2.internal"
Jul 20 19:56:23.971834 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:23.971813    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-191.ec2.internal\" not found"
Jul 20 19:56:23.991582 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.991552    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0c5e8c7baca191c444a3231e6958085d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-191.ec2.internal\" (UID: \"0c5e8c7baca191c444a3231e6958085d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-191.ec2.internal"
Jul 20 19:56:23.991670 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.991587    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c5e8c7baca191c444a3231e6958085d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-191.ec2.internal\" (UID: \"0c5e8c7baca191c444a3231e6958085d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-191.ec2.internal"
Jul 20 19:56:23.991670 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:23.991604    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/504046be7821a23dec7f8227b49cfc17-config\") pod \"kube-apiserver-proxy-ip-10-0-143-191.ec2.internal\" (UID: \"504046be7821a23dec7f8227b49cfc17\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-191.ec2.internal"
Jul 20 19:56:24.072626 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:24.072596    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-191.ec2.internal\" not found"
Jul 20 19:56:24.092062 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:24.092036    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0c5e8c7baca191c444a3231e6958085d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-191.ec2.internal\" (UID: \"0c5e8c7baca191c444a3231e6958085d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-191.ec2.internal"
Jul 20 19:56:24.092132 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:24.092064    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c5e8c7baca191c444a3231e6958085d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-191.ec2.internal\" (UID: \"0c5e8c7baca191c444a3231e6958085d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-191.ec2.internal"
Jul 20 19:56:24.092132 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:24.092081    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/504046be7821a23dec7f8227b49cfc17-config\") pod \"kube-apiserver-proxy-ip-10-0-143-191.ec2.internal\" (UID: \"504046be7821a23dec7f8227b49cfc17\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-191.ec2.internal"
Jul 20 19:56:24.092206 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:24.092132    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c5e8c7baca191c444a3231e6958085d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-191.ec2.internal\" (UID: \"0c5e8c7baca191c444a3231e6958085d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-191.ec2.internal"
Jul 20 19:56:24.092206 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:24.092134    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0c5e8c7baca191c444a3231e6958085d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-191.ec2.internal\" (UID: \"0c5e8c7baca191c444a3231e6958085d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-191.ec2.internal"
Jul 20 19:56:24.092206 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:24.092186    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/504046be7821a23dec7f8227b49cfc17-config\") pod \"kube-apiserver-proxy-ip-10-0-143-191.ec2.internal\" (UID: \"504046be7821a23dec7f8227b49cfc17\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-191.ec2.internal"
Jul 20 19:56:24.173426 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:24.173358    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-191.ec2.internal\" not found"
Jul 20 19:56:24.256848 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:24.256820    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-191.ec2.internal"
Jul 20 19:56:24.259985 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:24.259968    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-191.ec2.internal"
Jul 20 19:56:24.274210 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:24.274190    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-191.ec2.internal\" not found"
Jul 20 19:56:25.713764 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:25.713721    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-191.ec2.internal\" not found"
Jul 20 19:56:25.814374 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:25.814293    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-191.ec2.internal\" not found"
Jul 20 19:56:25.831497 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:25.831475    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 19:56:25.926323 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:25.926299    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-191.ec2.internal"
Jul 20 19:56:25.934133 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:25.934109    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 20 19:56:25.935604 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:25.935592    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-191.ec2.internal"
Jul 20 19:56:25.945384 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:25.945360    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 20 19:56:25.972541 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:25.972519    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 20 19:56:25.972924 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:25.972635    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 20 19:56:25.972924 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:25.972682    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 20 19:56:25.994724 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:25.994692    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 19:56:26.007132 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.007102    2580 apiserver.go:52] "Watching apiserver"
Jul 20 19:56:26.014427 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.014403    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 20 19:56:26.015300 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.015278    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-b94k5","openshift-multus/multus-jzsk9","openshift-multus/network-metrics-daemon-5778m","openshift-network-operator/iptables-alerter-59kk4","kube-system/global-pull-secret-syncer-xlc4n","openshift-cluster-node-tuning-operator/tuned-v4zzw","openshift-dns/node-resolver-nngvn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-191.ec2.internal","openshift-network-diagnostics/network-check-target-5gtt2","openshift-ovn-kubernetes/ovnkube-node-8ldg4","kube-system/konnectivity-agent-nb8cs","kube-system/kube-apiserver-proxy-ip-10-0-143-191.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf","openshift-image-registry/node-ca-w5555"]
Jul 20 19:56:26.017729 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.017706    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.019844 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.019808    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.020700 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.020550    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 20 19:56:26.020700 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.020563    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 20 19:56:26.020700 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.020554    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 20 19:56:26.020700 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.020613    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5sqpb\""
Jul 20 19:56:26.020947 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.020841    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 20 19:56:26.020947 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.020843    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 20 19:56:26.022251 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.022218    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 20 19:56:26.022340 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.022219    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ppbws\""
Jul 20 19:56:26.022473 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.022456    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:26.022549 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:26.022532    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5778m" podUID="c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1"
Jul 20 19:56:26.024888 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.024870    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-59kk4"
Jul 20 19:56:26.025514 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.025501    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 20 19:56:26.027200 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.027185    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 20 19:56:26.027491 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.027475    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 20 19:56:26.027554 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.027537    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5lgnk\""
Jul 20 19:56:26.027621 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.027606    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:26.027697 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:26.027676    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xlc4n" podUID="2fbbdeaa-171d-4032-9bd3-7bbd4c009388"
Jul 20 19:56:26.027741 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.027725    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 20 19:56:26.030297 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.030284    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.032643 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.032625    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-298nb\""
Jul 20 19:56:26.032776 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.032740    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 20 19:56:26.032865 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.032843    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nngvn"
Jul 20 19:56:26.033187 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.033171    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 20 19:56:26.035305 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035291    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 20 19:56:26.035399 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035294    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-multus-cni-dir\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.035399 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035351    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eddfe126-5be8-4f75-80bc-97f01e169ba4-cni-binary-copy\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.035399 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035381    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bfaea5fd-9480-4cd0-b133-217c1eb6069a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.035563 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035419    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrc57\" (UniqueName: \"kubernetes.io/projected/bfaea5fd-9480-4cd0-b133-217c1eb6069a-kube-api-access-rrc57\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.035563 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035445    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-host-run-netns\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.035563 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035479    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-host-var-lib-cni-multus\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.035563 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035504    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gddjn\" (UniqueName: \"kubernetes.io/projected/eddfe126-5be8-4f75-80bc-97f01e169ba4-kube-api-access-gddjn\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.035563 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035528    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-host-run-k8s-cni-cncf-io\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.035755 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035575    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-host-var-lib-kubelet\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.035755 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035623    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-multus-socket-dir-parent\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.035755 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035633    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vt98k\""
Jul 20 19:56:26.035755 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035654    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-host-var-lib-cni-bin\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.035755 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035679    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-hostroot\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.035755 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035680    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:26.035755 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035715    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8f2f4dc6-3966-4eff-aecc-06298645ecb5-iptables-alerter-script\") pod \"iptables-alerter-59kk4\" (UID: \"8f2f4dc6-3966-4eff-aecc-06298645ecb5\") " pod="openshift-network-operator/iptables-alerter-59kk4"
Jul 20 19:56:26.035755 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035740    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw9ch\" (UniqueName: \"kubernetes.io/projected/8f2f4dc6-3966-4eff-aecc-06298645ecb5-kube-api-access-kw9ch\") pod \"iptables-alerter-59kk4\" (UID: \"8f2f4dc6-3966-4eff-aecc-06298645ecb5\") " pod="openshift-network-operator/iptables-alerter-59kk4"
Jul 20 19:56:26.035755 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:26.035741    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gtt2" podUID="62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d"
Jul 20 19:56:26.036196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035765    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bfaea5fd-9480-4cd0-b133-217c1eb6069a-system-cni-dir\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.036196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035777    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 20 19:56:26.036196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035790    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-cnibin\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.036196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035812    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-os-release\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.036196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035834    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-multus-conf-dir\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.036196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035870    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/eddfe126-5be8-4f75-80bc-97f01e169ba4-multus-daemon-config\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.036196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035900    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csv84\" (UniqueName: \"kubernetes.io/projected/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-kube-api-access-csv84\") pod \"network-metrics-daemon-5778m\" (UID: \"c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1\") " pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:26.036196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035925    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8f2f4dc6-3966-4eff-aecc-06298645ecb5-host-slash\") pod \"iptables-alerter-59kk4\" (UID: \"8f2f4dc6-3966-4eff-aecc-06298645ecb5\") " pod="openshift-network-operator/iptables-alerter-59kk4"
Jul 20 19:56:26.036196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035951    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bfaea5fd-9480-4cd0-b133-217c1eb6069a-cnibin\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.036196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.035984    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bfaea5fd-9480-4cd0-b133-217c1eb6069a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.036196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.036018    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret\") pod \"global-pull-secret-syncer-xlc4n\" (UID: \"2fbbdeaa-171d-4032-9bd3-7bbd4c009388\") " pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:26.036196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.036051    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bfaea5fd-9480-4cd0-b133-217c1eb6069a-os-release\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.036196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.036075    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-system-cni-dir\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.036196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.036099    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-host-run-multus-certs\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.036196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.036121    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-etc-kubernetes\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.036196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.036144    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-kubelet-config\") pod \"global-pull-secret-syncer-xlc4n\" (UID: \"2fbbdeaa-171d-4032-9bd3-7bbd4c009388\") " pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:26.036196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.036168    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs\") pod \"network-metrics-daemon-5778m\" (UID: \"c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1\") " pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:26.036699 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.036190    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-dbus\") pod \"global-pull-secret-syncer-xlc4n\" (UID: \"2fbbdeaa-171d-4032-9bd3-7bbd4c009388\") " pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:26.036699 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.036211    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bfaea5fd-9480-4cd0-b133-217c1eb6069a-cni-binary-copy\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.036699 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.036236    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bfaea5fd-9480-4cd0-b133-217c1eb6069a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.038280 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.038260    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.038377 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.038340    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 20 19:56:26.040627 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.040587    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-19 19:51:25 +0000 UTC" deadline="2028-01-29 19:52:39.285266551 +0000 UTC"
Jul 20 19:56:26.040627 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.040625    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13391h56m13.244644511s"
Jul 20 19:56:26.041195 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.040872    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nb8cs"
Jul 20 19:56:26.043604 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.043588    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.044209 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.044186    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 20 19:56:26.044209 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.044201    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qmtnp\""
Jul 20 19:56:26.044608 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.044587    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 20 19:56:26.044704 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.044617    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 20 19:56:26.045034 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.045016    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 20 19:56:26.045034 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.045024    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 20 19:56:26.045205 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.045126    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 20 19:56:26.045396 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.045367    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 20 19:56:26.045537 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.045518    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 20 19:56:26.046681 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.046108    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4rzbm\""
Jul 20 19:56:26.046681 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.046365    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dvd6n\""
Jul 20 19:56:26.046681 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.046457    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w5555"
Jul 20 19:56:26.047028 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.047010    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 20 19:56:26.047710 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.047693    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 20 19:56:26.048043 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.048018    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 20 19:56:26.048896 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.048881    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-v5j9q\""
Jul 20 19:56:26.049185 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.049166    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 20 19:56:26.049185 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.049181    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 20 19:56:26.049401 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.049388    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 20 19:56:26.061414 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.061398    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2fnxs"
Jul 20 19:56:26.069547 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.069530    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2fnxs"
Jul 20 19:56:26.127322 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.127303    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 20 19:56:26.136839 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.136819    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1a0e6c85-e495-4711-a23e-acdb2c031587-ovnkube-script-lib\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.136943 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.136845    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/36752684-c032-4e36-bbe5-65480fc3d8d8-etc-selinux\") pod \"aws-ebs-csi-driver-node-wk7jf\" (UID: \"36752684-c032-4e36-bbe5-65480fc3d8d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.136943 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.136875    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sls9\" (UniqueName: \"kubernetes.io/projected/36752684-c032-4e36-bbe5-65480fc3d8d8-kube-api-access-9sls9\") pod \"aws-ebs-csi-driver-node-wk7jf\" (UID: \"36752684-c032-4e36-bbe5-65480fc3d8d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.136943 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.136901    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs\") pod \"network-metrics-daemon-5778m\" (UID: \"c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1\") " pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:26.136943 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.136917    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-dbus\") pod \"global-pull-secret-syncer-xlc4n\" (UID: \"2fbbdeaa-171d-4032-9bd3-7bbd4c009388\") " pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:26.136943 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.136934    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a164b5a7-e049-42a5-8f89-915704f82f68-tmp-dir\") pod \"node-resolver-nngvn\" (UID: \"a164b5a7-e049-42a5-8f89-915704f82f68\") " pod="openshift-dns/node-resolver-nngvn"
Jul 20 19:56:26.137196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.136949    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.137196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.136991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eddfe126-5be8-4f75-80bc-97f01e169ba4-cni-binary-copy\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.137196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137007    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-var-lib-openvswitch\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.137196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137021    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-node-log\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.137196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137064    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/36752684-c032-4e36-bbe5-65480fc3d8d8-socket-dir\") pod \"aws-ebs-csi-driver-node-wk7jf\" (UID: \"36752684-c032-4e36-bbe5-65480fc3d8d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.137196 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:26.137074    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 19:56:26.137196 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:26.137142    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs podName:c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1 nodeName:}" failed. No retries permitted until 2026-07-20 19:56:26.637120091 +0000 UTC m=+1.901190142 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs") pod "network-metrics-daemon-5778m" (UID: "c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 19:56:26.137196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gddjn\" (UniqueName: \"kubernetes.io/projected/eddfe126-5be8-4f75-80bc-97f01e169ba4-kube-api-access-gddjn\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.137555 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137183    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-dbus\") pod \"global-pull-secret-syncer-xlc4n\" (UID: \"2fbbdeaa-171d-4032-9bd3-7bbd4c009388\") " pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:26.137555 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137222    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k87vw\" (UniqueName: \"kubernetes.io/projected/b4c1d9fe-73ea-4091-9855-b9f6970d176e-kube-api-access-k87vw\") pod \"node-ca-w5555\" (UID: \"b4c1d9fe-73ea-4091-9855-b9f6970d176e\") " pod="openshift-image-registry/node-ca-w5555"
Jul 20 19:56:26.137555 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137277    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1a0e6c85-e495-4711-a23e-acdb2c031587-ovnkube-config\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.137555 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137314    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-host-run-k8s-cni-cncf-io\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.137555 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137340    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-888ww\" (UniqueName: \"kubernetes.io/projected/a164b5a7-e049-42a5-8f89-915704f82f68-kube-api-access-888ww\") pod \"node-resolver-nngvn\" (UID: \"a164b5a7-e049-42a5-8f89-915704f82f68\") " pod="openshift-dns/node-resolver-nngvn"
Jul 20 19:56:26.137555 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137366    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldtl2\" (UniqueName: \"kubernetes.io/projected/1a0e6c85-e495-4711-a23e-acdb2c031587-kube-api-access-ldtl2\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.137555 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137379    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-host-run-k8s-cni-cncf-io\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.137555 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137390    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/36752684-c032-4e36-bbe5-65480fc3d8d8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wk7jf\" (UID: \"36752684-c032-4e36-bbe5-65480fc3d8d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.137555 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137426    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/36752684-c032-4e36-bbe5-65480fc3d8d8-sys-fs\") pod \"aws-ebs-csi-driver-node-wk7jf\" (UID: \"36752684-c032-4e36-bbe5-65480fc3d8d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.137555 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137471    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-multus-socket-dir-parent\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.137555 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137498    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eddfe126-5be8-4f75-80bc-97f01e169ba4-cni-binary-copy\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.137555 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-hostroot\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.137555 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137520    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-multus-socket-dir-parent\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.137555 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137544    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-host-run-netns\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.138129 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137559    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-hostroot\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.138129 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137601    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-multus-conf-dir\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.138129 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csv84\" (UniqueName: \"kubernetes.io/projected/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-kube-api-access-csv84\") pod \"network-metrics-daemon-5778m\" (UID: \"c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1\") " pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:26.138129 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137645    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-multus-conf-dir\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.138129 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8f2f4dc6-3966-4eff-aecc-06298645ecb5-host-slash\") pod \"iptables-alerter-59kk4\" (UID: \"8f2f4dc6-3966-4eff-aecc-06298645ecb5\") " pod="openshift-network-operator/iptables-alerter-59kk4"
Jul 20 19:56:26.138129 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137689    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-run\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.138129 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137709    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-var-lib-kubelet\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.138129 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137721    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8f2f4dc6-3966-4eff-aecc-06298645ecb5-host-slash\") pod \"iptables-alerter-59kk4\" (UID: \"8f2f4dc6-3966-4eff-aecc-06298645ecb5\") " pod="openshift-network-operator/iptables-alerter-59kk4"
Jul 20 19:56:26.138129 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137731    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-tmp\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.138129 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137759    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/36752684-c032-4e36-bbe5-65480fc3d8d8-registration-dir\") pod \"aws-ebs-csi-driver-node-wk7jf\" (UID: \"36752684-c032-4e36-bbe5-65480fc3d8d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.138129 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137779    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-etc-sysconfig\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.138129 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137794    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-etc-systemd\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.138129 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137826    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-system-cni-dir\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.138129 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137860    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-host-run-multus-certs\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.138129 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137887    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-host-run-multus-certs\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.138129 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137906    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-system-cni-dir\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.138129 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137916    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-kubelet-config\") pod \"global-pull-secret-syncer-xlc4n\" (UID: \"2fbbdeaa-171d-4032-9bd3-7bbd4c009388\") " pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:26.138898 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.137945    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-etc-modprobe-d\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.138898 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138001    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-kubelet-config\") pod \"global-pull-secret-syncer-xlc4n\" (UID: \"2fbbdeaa-171d-4032-9bd3-7bbd4c009388\") " pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:26.138898 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138032    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrv42\" (UniqueName: \"kubernetes.io/projected/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-kube-api-access-xrv42\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.138898 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138055    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdmfr\" (UniqueName: \"kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr\") pod \"network-check-target-5gtt2\" (UID: \"62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d\") " pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:26.138898 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138071    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-log-socket\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.138898 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bfaea5fd-9480-4cd0-b133-217c1eb6069a-cni-binary-copy\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.138898 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138115    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bfaea5fd-9480-4cd0-b133-217c1eb6069a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.138898 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138145    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-sys\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.138898 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138171    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-host-slash\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.138898 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138195    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1a0e6c85-e495-4711-a23e-acdb2c031587-env-overrides\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.138898 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138231    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1a0e6c85-e495-4711-a23e-acdb2c031587-ovn-node-metrics-cert\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.138898 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-multus-cni-dir\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.138898 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138298    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bfaea5fd-9480-4cd0-b133-217c1eb6069a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.138898 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138323    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrc57\" (UniqueName: \"kubernetes.io/projected/bfaea5fd-9480-4cd0-b133-217c1eb6069a-kube-api-access-rrc57\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.138898 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138329    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-multus-cni-dir\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.138898 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-etc-sysctl-d\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.139598 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138370    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b4c1d9fe-73ea-4091-9855-b9f6970d176e-serviceca\") pod \"node-ca-w5555\" (UID: \"b4c1d9fe-73ea-4091-9855-b9f6970d176e\") " pod="openshift-image-registry/node-ca-w5555"
Jul 20 19:56:26.139598 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138391    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-run-systemd\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.139598 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138416    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-etc-openvswitch\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.139598 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138477    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-host-run-netns\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.139598 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138501    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-host-var-lib-cni-multus\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.139598 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138523    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-host\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.139598 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138558    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-host-run-netns\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.139598 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138567    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-host-var-lib-cni-multus\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.139598 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138603    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-host-var-lib-kubelet\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.139598 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138610    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bfaea5fd-9480-4cd0-b133-217c1eb6069a-cni-binary-copy\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.139598 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138620    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bfaea5fd-9480-4cd0-b133-217c1eb6069a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.139598 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138632    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/36752684-c032-4e36-bbe5-65480fc3d8d8-device-dir\") pod \"aws-ebs-csi-driver-node-wk7jf\" (UID: \"36752684-c032-4e36-bbe5-65480fc3d8d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.139598 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138665    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-host-var-lib-cni-bin\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.139598 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138670    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-host-var-lib-kubelet\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.139598 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138690    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8f2f4dc6-3966-4eff-aecc-06298645ecb5-iptables-alerter-script\") pod \"iptables-alerter-59kk4\" (UID: \"8f2f4dc6-3966-4eff-aecc-06298645ecb5\") " pod="openshift-network-operator/iptables-alerter-59kk4"
Jul 20 19:56:26.139598 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138715    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw9ch\" (UniqueName: \"kubernetes.io/projected/8f2f4dc6-3966-4eff-aecc-06298645ecb5-kube-api-access-kw9ch\") pod \"iptables-alerter-59kk4\" (UID: \"8f2f4dc6-3966-4eff-aecc-06298645ecb5\") " pod="openshift-network-operator/iptables-alerter-59kk4"
Jul 20 19:56:26.139598 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138727    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bfaea5fd-9480-4cd0-b133-217c1eb6069a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.140168 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138740    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bfaea5fd-9480-4cd0-b133-217c1eb6069a-system-cni-dir\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.140168 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138730    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-host-var-lib-cni-bin\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.140168 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138768    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-etc-kubernetes\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.140168 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138779    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bfaea5fd-9480-4cd0-b133-217c1eb6069a-system-cni-dir\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.140168 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138792    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d518f591-565e-4ac1-b5d1-f88311868032-agent-certs\") pod \"konnectivity-agent-nb8cs\" (UID: \"d518f591-565e-4ac1-b5d1-f88311868032\") " pod="kube-system/konnectivity-agent-nb8cs"
Jul 20 19:56:26.140168 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138816    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a164b5a7-e049-42a5-8f89-915704f82f68-hosts-file\") pod \"node-resolver-nngvn\" (UID: \"a164b5a7-e049-42a5-8f89-915704f82f68\") " pod="openshift-dns/node-resolver-nngvn"
Jul 20 19:56:26.140168 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138842    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-cnibin\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.140168 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138863    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-os-release\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.140168 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138887    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/eddfe126-5be8-4f75-80bc-97f01e169ba4-multus-daemon-config\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.140168 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138895    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-cnibin\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.140168 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138908    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bfaea5fd-9480-4cd0-b133-217c1eb6069a-cnibin\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.140168 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138933    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bfaea5fd-9480-4cd0-b133-217c1eb6069a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.140168 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138941    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-os-release\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.140168 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138970    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-etc-sysctl-conf\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.140168 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138994    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-run-openvswitch\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.140168 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.138992    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bfaea5fd-9480-4cd0-b133-217c1eb6069a-cnibin\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.140168 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.139030    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-run-ovn\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.140789 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.139076    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret\") pod \"global-pull-secret-syncer-xlc4n\" (UID: \"2fbbdeaa-171d-4032-9bd3-7bbd4c009388\") " pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:26.140789 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.139088    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bfaea5fd-9480-4cd0-b133-217c1eb6069a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.140789 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.139102    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bfaea5fd-9480-4cd0-b133-217c1eb6069a-os-release\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.140789 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.139127    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-etc-tuned\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.140789 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.139163    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bfaea5fd-9480-4cd0-b133-217c1eb6069a-os-release\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.140789 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.139156    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d518f591-565e-4ac1-b5d1-f88311868032-konnectivity-ca\") pod \"konnectivity-agent-nb8cs\" (UID: \"d518f591-565e-4ac1-b5d1-f88311868032\") " pod="kube-system/konnectivity-agent-nb8cs"
Jul 20 19:56:26.140789 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:26.139193    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 19:56:26.140789 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:26.139242    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret podName:2fbbdeaa-171d-4032-9bd3-7bbd4c009388 nodeName:}" failed. No retries permitted until 2026-07-20 19:56:26.639226816 +0000 UTC m=+1.903296845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret") pod "global-pull-secret-syncer-xlc4n" (UID: "2fbbdeaa-171d-4032-9bd3-7bbd4c009388") : object "kube-system"/"original-pull-secret" not registered
Jul 20 19:56:26.140789 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.139198    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-host-kubelet\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.140789 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.139190    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8f2f4dc6-3966-4eff-aecc-06298645ecb5-iptables-alerter-script\") pod \"iptables-alerter-59kk4\" (UID: \"8f2f4dc6-3966-4eff-aecc-06298645ecb5\") " pod="openshift-network-operator/iptables-alerter-59kk4"
Jul 20 19:56:26.140789 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.139281    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-systemd-units\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.140789 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.139319    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-host-run-ovn-kubernetes\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.140789 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.139325    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/eddfe126-5be8-4f75-80bc-97f01e169ba4-multus-daemon-config\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.140789 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.139334    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-host-cni-bin\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.140789 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.139372    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-etc-kubernetes\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.140789 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.139398    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-lib-modules\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.140789 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.139447    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eddfe126-5be8-4f75-80bc-97f01e169ba4-etc-kubernetes\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.141583 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.139454    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b4c1d9fe-73ea-4091-9855-b9f6970d176e-host\") pod \"node-ca-w5555\" (UID: \"b4c1d9fe-73ea-4091-9855-b9f6970d176e\") " pod="openshift-image-registry/node-ca-w5555"
Jul 20 19:56:26.141583 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.139488    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-host-cni-netd\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.147189 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.147169    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 20 19:56:26.150415 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.150393    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gddjn\" (UniqueName: \"kubernetes.io/projected/eddfe126-5be8-4f75-80bc-97f01e169ba4-kube-api-access-gddjn\") pod \"multus-jzsk9\" (UID: \"eddfe126-5be8-4f75-80bc-97f01e169ba4\") " pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.150492 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.150472    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrc57\" (UniqueName: \"kubernetes.io/projected/bfaea5fd-9480-4cd0-b133-217c1eb6069a-kube-api-access-rrc57\") pod \"multus-additional-cni-plugins-b94k5\" (UID: \"bfaea5fd-9480-4cd0-b133-217c1eb6069a\") " pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.150534 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.150522    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csv84\" (UniqueName: \"kubernetes.io/projected/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-kube-api-access-csv84\") pod \"network-metrics-daemon-5778m\" (UID: \"c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1\") " pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:26.157780 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.157758    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw9ch\" (UniqueName: \"kubernetes.io/projected/8f2f4dc6-3966-4eff-aecc-06298645ecb5-kube-api-access-kw9ch\") pod \"iptables-alerter-59kk4\" (UID: \"8f2f4dc6-3966-4eff-aecc-06298645ecb5\") " pod="openshift-network-operator/iptables-alerter-59kk4"
Jul 20 19:56:26.179627 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:26.177711    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod504046be7821a23dec7f8227b49cfc17.slice/crio-36e6b5e6700f807c726fd52d47d9d2af096f553e298c02c74c33df821bdb2191 WatchSource:0}: Error finding container 36e6b5e6700f807c726fd52d47d9d2af096f553e298c02c74c33df821bdb2191: Status 404 returned error can't find the container with id 36e6b5e6700f807c726fd52d47d9d2af096f553e298c02c74c33df821bdb2191
Jul 20 19:56:26.184728 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.184712    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 19:56:26.190158 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:26.190136    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c5e8c7baca191c444a3231e6958085d.slice/crio-d1bbf5d220e4d6287aa064199b794a554c96bb75de5f42ee189c33f40ce0c230 WatchSource:0}: Error finding container d1bbf5d220e4d6287aa064199b794a554c96bb75de5f42ee189c33f40ce0c230: Status 404 returned error can't find the container with id d1bbf5d220e4d6287aa064199b794a554c96bb75de5f42ee189c33f40ce0c230
Jul 20 19:56:26.240437 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240412    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-host-run-netns\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.240437 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240441    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-run\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.240643 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240457    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-var-lib-kubelet\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.240643 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240471    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-tmp\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.240643 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240511    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/36752684-c032-4e36-bbe5-65480fc3d8d8-registration-dir\") pod \"aws-ebs-csi-driver-node-wk7jf\" (UID: \"36752684-c032-4e36-bbe5-65480fc3d8d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.240643 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240532    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-run\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.240643 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240532    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-host-run-netns\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.240643 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240549    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-etc-sysconfig\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.240643 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240575    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-etc-systemd\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.240643 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240591    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/36752684-c032-4e36-bbe5-65480fc3d8d8-registration-dir\") pod \"aws-ebs-csi-driver-node-wk7jf\" (UID: \"36752684-c032-4e36-bbe5-65480fc3d8d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.240643 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240600    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-etc-modprobe-d\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.240643 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240640    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-etc-systemd\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.241147 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240637    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-etc-sysconfig\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.241147 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrv42\" (UniqueName: \"kubernetes.io/projected/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-kube-api-access-xrv42\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.241147 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240600    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-var-lib-kubelet\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.241147 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240683    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdmfr\" (UniqueName: \"kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr\") pod \"network-check-target-5gtt2\" (UID: \"62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d\") " pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:26.241147 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240714    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-etc-modprobe-d\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.241147 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240741    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-log-socket\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.241147 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240772    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-sys\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.241147 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240796    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-host-slash\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.241147 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240802    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-log-socket\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.241147 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240820    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1a0e6c85-e495-4711-a23e-acdb2c031587-env-overrides\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.241147 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240826    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-sys\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.241147 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240845    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-host-slash\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.241147 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240863    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1a0e6c85-e495-4711-a23e-acdb2c031587-ovn-node-metrics-cert\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.241147 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240904    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-etc-sysctl-d\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.241147 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240928    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b4c1d9fe-73ea-4091-9855-b9f6970d176e-serviceca\") pod \"node-ca-w5555\" (UID: \"b4c1d9fe-73ea-4091-9855-b9f6970d176e\") " pod="openshift-image-registry/node-ca-w5555"
Jul 20 19:56:26.241147 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240949    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-run-systemd\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.241147 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.240989    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-etc-openvswitch\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.241147 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241015    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-host\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.241990 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241038    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/36752684-c032-4e36-bbe5-65480fc3d8d8-device-dir\") pod \"aws-ebs-csi-driver-node-wk7jf\" (UID: \"36752684-c032-4e36-bbe5-65480fc3d8d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.241990 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241064    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-etc-kubernetes\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.241990 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241075    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-etc-sysctl-d\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.241990 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d518f591-565e-4ac1-b5d1-f88311868032-agent-certs\") pod \"konnectivity-agent-nb8cs\" (UID: \"d518f591-565e-4ac1-b5d1-f88311868032\") " pod="kube-system/konnectivity-agent-nb8cs"
Jul 20 19:56:26.241990 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241111    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a164b5a7-e049-42a5-8f89-915704f82f68-hosts-file\") pod \"node-resolver-nngvn\" (UID: \"a164b5a7-e049-42a5-8f89-915704f82f68\") " pod="openshift-dns/node-resolver-nngvn"
Jul 20 19:56:26.241990 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241110    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-run-systemd\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.241990 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241133    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/36752684-c032-4e36-bbe5-65480fc3d8d8-device-dir\") pod \"aws-ebs-csi-driver-node-wk7jf\" (UID: \"36752684-c032-4e36-bbe5-65480fc3d8d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.241990 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241139    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-etc-sysctl-conf\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.241990 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241178    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-host\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.241990 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241180    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-run-openvswitch\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.241990 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241221    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-run-ovn\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.241990 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241225    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-etc-kubernetes\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.241990 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-etc-tuned\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.241990 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241270    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-etc-sysctl-conf\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.241990 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241288    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d518f591-565e-4ac1-b5d1-f88311868032-konnectivity-ca\") pod \"konnectivity-agent-nb8cs\" (UID: \"d518f591-565e-4ac1-b5d1-f88311868032\") " pod="kube-system/konnectivity-agent-nb8cs"
Jul 20 19:56:26.241990 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241291    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-run-ovn\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.241990 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241316    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-host-kubelet\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.241990 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241342    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-systemd-units\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.242802 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241370    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-host-run-ovn-kubernetes\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.242802 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241386    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b4c1d9fe-73ea-4091-9855-b9f6970d176e-serviceca\") pod \"node-ca-w5555\" (UID: \"b4c1d9fe-73ea-4091-9855-b9f6970d176e\") " pod="openshift-image-registry/node-ca-w5555"
Jul 20 19:56:26.242802 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241395    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-host-cni-bin\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.242802 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241261    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-run-openvswitch\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.242802 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241430    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-lib-modules\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.242802 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241452    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a164b5a7-e049-42a5-8f89-915704f82f68-hosts-file\") pod \"node-resolver-nngvn\" (UID: \"a164b5a7-e049-42a5-8f89-915704f82f68\") " pod="openshift-dns/node-resolver-nngvn"
Jul 20 19:56:26.242802 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241453    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b4c1d9fe-73ea-4091-9855-b9f6970d176e-host\") pod \"node-ca-w5555\" (UID: \"b4c1d9fe-73ea-4091-9855-b9f6970d176e\") " pod="openshift-image-registry/node-ca-w5555"
Jul 20 19:56:26.242802 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241486    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b4c1d9fe-73ea-4091-9855-b9f6970d176e-host\") pod \"node-ca-w5555\" (UID: \"b4c1d9fe-73ea-4091-9855-b9f6970d176e\") " pod="openshift-image-registry/node-ca-w5555"
Jul 20 19:56:26.242802 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241500    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-host-cni-netd\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.242802 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241526    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-host-run-ovn-kubernetes\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.242802 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241525    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1a0e6c85-e495-4711-a23e-acdb2c031587-ovnkube-script-lib\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.242802 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241563    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/36752684-c032-4e36-bbe5-65480fc3d8d8-etc-selinux\") pod \"aws-ebs-csi-driver-node-wk7jf\" (UID: \"36752684-c032-4e36-bbe5-65480fc3d8d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.242802 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241589    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sls9\" (UniqueName: \"kubernetes.io/projected/36752684-c032-4e36-bbe5-65480fc3d8d8-kube-api-access-9sls9\") pod \"aws-ebs-csi-driver-node-wk7jf\" (UID: \"36752684-c032-4e36-bbe5-65480fc3d8d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.242802 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a164b5a7-e049-42a5-8f89-915704f82f68-tmp-dir\") pod \"node-resolver-nngvn\" (UID: \"a164b5a7-e049-42a5-8f89-915704f82f68\") " pod="openshift-dns/node-resolver-nngvn"
Jul 20 19:56:26.242802 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241654    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.242802 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241679    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-var-lib-openvswitch\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.242802 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241701    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-node-log\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.243317 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241723    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/36752684-c032-4e36-bbe5-65480fc3d8d8-socket-dir\") pod \"aws-ebs-csi-driver-node-wk7jf\" (UID: \"36752684-c032-4e36-bbe5-65480fc3d8d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.243317 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241751    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k87vw\" (UniqueName: \"kubernetes.io/projected/b4c1d9fe-73ea-4091-9855-b9f6970d176e-kube-api-access-k87vw\") pod \"node-ca-w5555\" (UID: \"b4c1d9fe-73ea-4091-9855-b9f6970d176e\") " pod="openshift-image-registry/node-ca-w5555"
Jul 20 19:56:26.243317 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241774    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1a0e6c85-e495-4711-a23e-acdb2c031587-ovnkube-config\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.243317 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241394    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1a0e6c85-e495-4711-a23e-acdb2c031587-env-overrides\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.243317 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241825    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-etc-openvswitch\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.243317 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241874    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-host-cni-bin\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.243317 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241908    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-lib-modules\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.243317 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.241989    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1a0e6c85-e495-4711-a23e-acdb2c031587-ovnkube-script-lib\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.243317 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.242001    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/36752684-c032-4e36-bbe5-65480fc3d8d8-etc-selinux\") pod \"aws-ebs-csi-driver-node-wk7jf\" (UID: \"36752684-c032-4e36-bbe5-65480fc3d8d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.243317 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.242039    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-systemd-units\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.243317 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.242089    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-host-cni-netd\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.243317 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.242131    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-var-lib-openvswitch\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.243317 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.242228    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-node-log\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.243317 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.242339    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/36752684-c032-4e36-bbe5-65480fc3d8d8-socket-dir\") pod \"aws-ebs-csi-driver-node-wk7jf\" (UID: \"36752684-c032-4e36-bbe5-65480fc3d8d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.243317 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.242410    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a164b5a7-e049-42a5-8f89-915704f82f68-tmp-dir\") pod \"node-resolver-nngvn\" (UID: \"a164b5a7-e049-42a5-8f89-915704f82f68\") " pod="openshift-dns/node-resolver-nngvn"
Jul 20 19:56:26.243317 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.242416    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d518f591-565e-4ac1-b5d1-f88311868032-konnectivity-ca\") pod \"konnectivity-agent-nb8cs\" (UID: \"d518f591-565e-4ac1-b5d1-f88311868032\") " pod="kube-system/konnectivity-agent-nb8cs"
Jul 20 19:56:26.243317 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.242476    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-host-kubelet\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.243798 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.242487    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a0e6c85-e495-4711-a23e-acdb2c031587-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.243798 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.242507    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-888ww\" (UniqueName: \"kubernetes.io/projected/a164b5a7-e049-42a5-8f89-915704f82f68-kube-api-access-888ww\") pod \"node-resolver-nngvn\" (UID: \"a164b5a7-e049-42a5-8f89-915704f82f68\") " pod="openshift-dns/node-resolver-nngvn"
Jul 20 19:56:26.243798 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.242536    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldtl2\" (UniqueName: \"kubernetes.io/projected/1a0e6c85-e495-4711-a23e-acdb2c031587-kube-api-access-ldtl2\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.243798 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.242560    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/36752684-c032-4e36-bbe5-65480fc3d8d8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wk7jf\" (UID: \"36752684-c032-4e36-bbe5-65480fc3d8d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.243798 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.242582    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/36752684-c032-4e36-bbe5-65480fc3d8d8-sys-fs\") pod \"aws-ebs-csi-driver-node-wk7jf\" (UID: \"36752684-c032-4e36-bbe5-65480fc3d8d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.243798 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.242652    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/36752684-c032-4e36-bbe5-65480fc3d8d8-sys-fs\") pod \"aws-ebs-csi-driver-node-wk7jf\" (UID: \"36752684-c032-4e36-bbe5-65480fc3d8d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.243798 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.242698    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/36752684-c032-4e36-bbe5-65480fc3d8d8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wk7jf\" (UID: \"36752684-c032-4e36-bbe5-65480fc3d8d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.243798 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.242900    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1a0e6c85-e495-4711-a23e-acdb2c031587-ovnkube-config\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.243798 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.243124    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-tmp\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.243798 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.243433    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1a0e6c85-e495-4711-a23e-acdb2c031587-ovn-node-metrics-cert\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.243798 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.243497    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-etc-tuned\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.243798 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.243681    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d518f591-565e-4ac1-b5d1-f88311868032-agent-certs\") pod \"konnectivity-agent-nb8cs\" (UID: \"d518f591-565e-4ac1-b5d1-f88311868032\") " pod="kube-system/konnectivity-agent-nb8cs"
Jul 20 19:56:26.254106 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:26.254088    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 19:56:26.254106 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:26.254106    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 19:56:26.254250 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:26.254120    2580 projected.go:196] Error preparing data for projected volume kube-api-access-tdmfr for pod openshift-network-diagnostics/network-check-target-5gtt2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 19:56:26.254250 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:26.254172    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr podName:62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d nodeName:}" failed. No retries permitted until 2026-07-20 19:56:26.754159134 +0000 UTC m=+2.018229159 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tdmfr" (UniqueName: "kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr") pod "network-check-target-5gtt2" (UID: "62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 19:56:26.256434 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.256410    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrv42\" (UniqueName: \"kubernetes.io/projected/d2b375ee-5209-4c9d-9ae8-baa440fb4f03-kube-api-access-xrv42\") pod \"tuned-v4zzw\" (UID: \"d2b375ee-5209-4c9d-9ae8-baa440fb4f03\") " pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.257266 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.257244    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k87vw\" (UniqueName: \"kubernetes.io/projected/b4c1d9fe-73ea-4091-9855-b9f6970d176e-kube-api-access-k87vw\") pod \"node-ca-w5555\" (UID: \"b4c1d9fe-73ea-4091-9855-b9f6970d176e\") " pod="openshift-image-registry/node-ca-w5555"
Jul 20 19:56:26.257370 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.257253    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sls9\" (UniqueName: \"kubernetes.io/projected/36752684-c032-4e36-bbe5-65480fc3d8d8-kube-api-access-9sls9\") pod \"aws-ebs-csi-driver-node-wk7jf\" (UID: \"36752684-c032-4e36-bbe5-65480fc3d8d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.257370 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.257318    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-888ww\" (UniqueName: \"kubernetes.io/projected/a164b5a7-e049-42a5-8f89-915704f82f68-kube-api-access-888ww\") pod \"node-resolver-nngvn\" (UID: \"a164b5a7-e049-42a5-8f89-915704f82f68\") " pod="openshift-dns/node-resolver-nngvn"
Jul 20 19:56:26.257370 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.257351    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldtl2\" (UniqueName: \"kubernetes.io/projected/1a0e6c85-e495-4711-a23e-acdb2c031587-kube-api-access-ldtl2\") pod \"ovnkube-node-8ldg4\" (UID: \"1a0e6c85-e495-4711-a23e-acdb2c031587\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.354638 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.354566    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b94k5"
Jul 20 19:56:26.357698 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.357678    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jzsk9"
Jul 20 19:56:26.366485 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:26.366461    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfaea5fd_9480_4cd0_b133_217c1eb6069a.slice/crio-7af909dba626b4d4733d75118c9a32d831f150161747e21c7c51c1bea459c6b9 WatchSource:0}: Error finding container 7af909dba626b4d4733d75118c9a32d831f150161747e21c7c51c1bea459c6b9: Status 404 returned error can't find the container with id 7af909dba626b4d4733d75118c9a32d831f150161747e21c7c51c1bea459c6b9
Jul 20 19:56:26.367185 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:26.367163    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeddfe126_5be8_4f75_80bc_97f01e169ba4.slice/crio-8be17c9520acfad060be72e5d0c48f4aad72e6b169122d5193cbc6d902872b9e WatchSource:0}: Error finding container 8be17c9520acfad060be72e5d0c48f4aad72e6b169122d5193cbc6d902872b9e: Status 404 returned error can't find the container with id 8be17c9520acfad060be72e5d0c48f4aad72e6b169122d5193cbc6d902872b9e
Jul 20 19:56:26.373400 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.373379    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-59kk4"
Jul 20 19:56:26.377037 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.376919    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v4zzw"
Jul 20 19:56:26.382973 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:26.382927    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f2f4dc6_3966_4eff_aecc_06298645ecb5.slice/crio-4be19747c1af378a4683e6968447b49a2cb6b682c662b373dbe84787b0413ef2 WatchSource:0}: Error finding container 4be19747c1af378a4683e6968447b49a2cb6b682c662b373dbe84787b0413ef2: Status 404 returned error can't find the container with id 4be19747c1af378a4683e6968447b49a2cb6b682c662b373dbe84787b0413ef2
Jul 20 19:56:26.386711 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:26.386609    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2b375ee_5209_4c9d_9ae8_baa440fb4f03.slice/crio-8e0f242700e297bcf146385f7a5eb58dd072e1ebe8caf0e8c00f5033341a636e WatchSource:0}: Error finding container 8e0f242700e297bcf146385f7a5eb58dd072e1ebe8caf0e8c00f5033341a636e: Status 404 returned error can't find the container with id 8e0f242700e297bcf146385f7a5eb58dd072e1ebe8caf0e8c00f5033341a636e
Jul 20 19:56:26.400789 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.400771    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nngvn"
Jul 20 19:56:26.409425 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:26.409398    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda164b5a7_e049_42a5_8f89_915704f82f68.slice/crio-80eb964a7eb7299ab1458a997e8d8aadc1303c99c7f3861c337cd1e803a50224 WatchSource:0}: Error finding container 80eb964a7eb7299ab1458a997e8d8aadc1303c99c7f3861c337cd1e803a50224: Status 404 returned error can't find the container with id 80eb964a7eb7299ab1458a997e8d8aadc1303c99c7f3861c337cd1e803a50224
Jul 20 19:56:26.421393 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.421377    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:26.427338 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.427319    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nb8cs"
Jul 20 19:56:26.429171 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:26.429149    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a0e6c85_e495_4711_a23e_acdb2c031587.slice/crio-efebef3370fdfae2dc330fc18f2db977f2ac33900a61a8da274e0712c3ded419 WatchSource:0}: Error finding container efebef3370fdfae2dc330fc18f2db977f2ac33900a61a8da274e0712c3ded419: Status 404 returned error can't find the container with id efebef3370fdfae2dc330fc18f2db977f2ac33900a61a8da274e0712c3ded419
Jul 20 19:56:26.434679 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:26.434660    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd518f591_565e_4ac1_b5d1_f88311868032.slice/crio-8f746d3c0c7e75506f43b12d9fe1dfd455af3670c78e9afc1bc1435a0e82416c WatchSource:0}: Error finding container 8f746d3c0c7e75506f43b12d9fe1dfd455af3670c78e9afc1bc1435a0e82416c: Status 404 returned error can't find the container with id 8f746d3c0c7e75506f43b12d9fe1dfd455af3670c78e9afc1bc1435a0e82416c
Jul 20 19:56:26.452079 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.452061    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf"
Jul 20 19:56:26.459377 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.459360    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 19:56:26.461488 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:26.461468    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36752684_c032_4e36_bbe5_65480fc3d8d8.slice/crio-e06a507c3ab368f295c69334e869ec728963950d133d1804ae316139b6f34f18 WatchSource:0}: Error finding container e06a507c3ab368f295c69334e869ec728963950d133d1804ae316139b6f34f18: Status 404 returned error can't find the container with id e06a507c3ab368f295c69334e869ec728963950d133d1804ae316139b6f34f18
Jul 20 19:56:26.462020 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.462007    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w5555"
Jul 20 19:56:26.470655 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:56:26.470636    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4c1d9fe_73ea_4091_9855_b9f6970d176e.slice/crio-8fb4becbf82ff6c03666ed5af90be8317e749031ed6faeb740bb219663147d38 WatchSource:0}: Error finding container 8fb4becbf82ff6c03666ed5af90be8317e749031ed6faeb740bb219663147d38: Status 404 returned error can't find the container with id 8fb4becbf82ff6c03666ed5af90be8317e749031ed6faeb740bb219663147d38
Jul 20 19:56:26.645013 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.644910    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret\") pod \"global-pull-secret-syncer-xlc4n\" (UID: \"2fbbdeaa-171d-4032-9bd3-7bbd4c009388\") " pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:26.645013 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.644973    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs\") pod \"network-metrics-daemon-5778m\" (UID: \"c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1\") " pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:26.645208 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:26.645075    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 19:56:26.645208 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:26.645079    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 19:56:26.645208 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:26.645129    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs podName:c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1 nodeName:}" failed. No retries permitted until 2026-07-20 19:56:27.645112774 +0000 UTC m=+2.909182800 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs") pod "network-metrics-daemon-5778m" (UID: "c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 19:56:26.645208 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:26.645143    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret podName:2fbbdeaa-171d-4032-9bd3-7bbd4c009388 nodeName:}" failed. No retries permitted until 2026-07-20 19:56:27.64513697 +0000 UTC m=+2.909206994 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret") pod "global-pull-secret-syncer-xlc4n" (UID: "2fbbdeaa-171d-4032-9bd3-7bbd4c009388") : object "kube-system"/"original-pull-secret" not registered
Jul 20 19:56:26.846911 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:26.846876    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdmfr\" (UniqueName: \"kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr\") pod \"network-check-target-5gtt2\" (UID: \"62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d\") " pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:26.847086 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:26.847042    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 19:56:26.847086 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:26.847066    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 19:56:26.847086 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:26.847078    2580 projected.go:196] Error preparing data for projected volume kube-api-access-tdmfr for pod openshift-network-diagnostics/network-check-target-5gtt2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 19:56:26.847263 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:26.847132    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr podName:62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d nodeName:}" failed. No retries permitted until 2026-07-20 19:56:27.847115044 +0000 UTC m=+3.111185072 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdmfr" (UniqueName: "kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr") pod "network-check-target-5gtt2" (UID: "62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 19:56:27.073052 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:27.072876    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 19:51:26 +0000 UTC" deadline="2028-04-08 00:40:23.121084156 +0000 UTC"
Jul 20 19:56:27.073052 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:27.072933    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15052h43m56.048156022s"
Jul 20 19:56:27.126276 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:27.126245    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 19:56:27.201784 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:27.201659    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nb8cs" event={"ID":"d518f591-565e-4ac1-b5d1-f88311868032","Type":"ContainerStarted","Data":"8f746d3c0c7e75506f43b12d9fe1dfd455af3670c78e9afc1bc1435a0e82416c"}
Jul 20 19:56:27.213547 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:27.213412    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4" event={"ID":"1a0e6c85-e495-4711-a23e-acdb2c031587","Type":"ContainerStarted","Data":"efebef3370fdfae2dc330fc18f2db977f2ac33900a61a8da274e0712c3ded419"}
Jul 20 19:56:27.234586 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:27.234537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nngvn" event={"ID":"a164b5a7-e049-42a5-8f89-915704f82f68","Type":"ContainerStarted","Data":"80eb964a7eb7299ab1458a997e8d8aadc1303c99c7f3861c337cd1e803a50224"}
Jul 20 19:56:27.251390 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:27.251354    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jzsk9" event={"ID":"eddfe126-5be8-4f75-80bc-97f01e169ba4","Type":"ContainerStarted","Data":"8be17c9520acfad060be72e5d0c48f4aad72e6b169122d5193cbc6d902872b9e"}
Jul 20 19:56:27.266934 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:27.266897    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b94k5" event={"ID":"bfaea5fd-9480-4cd0-b133-217c1eb6069a","Type":"ContainerStarted","Data":"7af909dba626b4d4733d75118c9a32d831f150161747e21c7c51c1bea459c6b9"}
Jul 20 19:56:27.284017 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:27.283982    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-191.ec2.internal" event={"ID":"0c5e8c7baca191c444a3231e6958085d","Type":"ContainerStarted","Data":"d1bbf5d220e4d6287aa064199b794a554c96bb75de5f42ee189c33f40ce0c230"}
Jul 20 19:56:27.312867 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:27.312819    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-191.ec2.internal" event={"ID":"504046be7821a23dec7f8227b49cfc17","Type":"ContainerStarted","Data":"36e6b5e6700f807c726fd52d47d9d2af096f553e298c02c74c33df821bdb2191"}
Jul 20 19:56:27.330875 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:27.330793    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w5555" event={"ID":"b4c1d9fe-73ea-4091-9855-b9f6970d176e","Type":"ContainerStarted","Data":"8fb4becbf82ff6c03666ed5af90be8317e749031ed6faeb740bb219663147d38"}
Jul 20 19:56:27.346836 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:27.346757    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf" event={"ID":"36752684-c032-4e36-bbe5-65480fc3d8d8","Type":"ContainerStarted","Data":"e06a507c3ab368f295c69334e869ec728963950d133d1804ae316139b6f34f18"}
Jul 20 19:56:27.359673 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:27.359637    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v4zzw" event={"ID":"d2b375ee-5209-4c9d-9ae8-baa440fb4f03","Type":"ContainerStarted","Data":"8e0f242700e297bcf146385f7a5eb58dd072e1ebe8caf0e8c00f5033341a636e"}
Jul 20 19:56:27.387478 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:27.387439    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-59kk4" event={"ID":"8f2f4dc6-3966-4eff-aecc-06298645ecb5","Type":"ContainerStarted","Data":"4be19747c1af378a4683e6968447b49a2cb6b682c662b373dbe84787b0413ef2"}
Jul 20 19:56:27.653761 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:27.653677    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret\") pod \"global-pull-secret-syncer-xlc4n\" (UID: \"2fbbdeaa-171d-4032-9bd3-7bbd4c009388\") " pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:27.653761 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:27.653733    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs\") pod \"network-metrics-daemon-5778m\" (UID: \"c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1\") " pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:27.653998 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:27.653890    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 19:56:27.653998 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:27.653952    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs podName:c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1 nodeName:}" failed. No retries permitted until 2026-07-20 19:56:29.653932049 +0000 UTC m=+4.918002075 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs") pod "network-metrics-daemon-5778m" (UID: "c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 19:56:27.654419 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:27.654398    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 19:56:27.654540 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:27.654464    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret podName:2fbbdeaa-171d-4032-9bd3-7bbd4c009388 nodeName:}" failed. No retries permitted until 2026-07-20 19:56:29.654446261 +0000 UTC m=+4.918516290 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret") pod "global-pull-secret-syncer-xlc4n" (UID: "2fbbdeaa-171d-4032-9bd3-7bbd4c009388") : object "kube-system"/"original-pull-secret" not registered
Jul 20 19:56:27.854750 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:27.854630    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdmfr\" (UniqueName: \"kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr\") pod \"network-check-target-5gtt2\" (UID: \"62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d\") " pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:27.854919 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:27.854872    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 19:56:27.854919 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:27.854892    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 19:56:27.854919 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:27.854907    2580 projected.go:196] Error preparing data for projected volume kube-api-access-tdmfr for pod openshift-network-diagnostics/network-check-target-5gtt2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 19:56:27.855177 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:27.854986    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr podName:62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d nodeName:}" failed. No retries permitted until 2026-07-20 19:56:29.85494887 +0000 UTC m=+5.119018896 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdmfr" (UniqueName: "kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr") pod "network-check-target-5gtt2" (UID: "62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 19:56:28.074038 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:28.073915    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 19:51:26 +0000 UTC" deadline="2028-01-14 15:15:59.79798949 +0000 UTC"
Jul 20 19:56:28.074485 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:28.074215    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13027h19m31.724037304s"
Jul 20 19:56:28.170797 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:28.170752    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:28.171038 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:28.170878    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gtt2" podUID="62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d"
Jul 20 19:56:28.171322 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:28.171301    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:28.171435 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:28.171415    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5778m" podUID="c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1"
Jul 20 19:56:28.171512 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:28.171499    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:28.171597 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:28.171579    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xlc4n" podUID="2fbbdeaa-171d-4032-9bd3-7bbd4c009388"
Jul 20 19:56:28.472501 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:28.472420    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 19:56:29.673564 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:29.673480    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret\") pod \"global-pull-secret-syncer-xlc4n\" (UID: \"2fbbdeaa-171d-4032-9bd3-7bbd4c009388\") " pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:29.673564 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:29.673534    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs\") pod \"network-metrics-daemon-5778m\" (UID: \"c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1\") " pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:29.674088 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:29.673680    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 19:56:29.674088 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:29.673746    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs podName:c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1 nodeName:}" failed. No retries permitted until 2026-07-20 19:56:33.673727419 +0000 UTC m=+8.937797456 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs") pod "network-metrics-daemon-5778m" (UID: "c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 19:56:29.674204 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:29.674162    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 19:56:29.674262 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:29.674218    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret podName:2fbbdeaa-171d-4032-9bd3-7bbd4c009388 nodeName:}" failed. No retries permitted until 2026-07-20 19:56:33.674203241 +0000 UTC m=+8.938273271 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret") pod "global-pull-secret-syncer-xlc4n" (UID: "2fbbdeaa-171d-4032-9bd3-7bbd4c009388") : object "kube-system"/"original-pull-secret" not registered
Jul 20 19:56:29.876237 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:29.876099    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdmfr\" (UniqueName: \"kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr\") pod \"network-check-target-5gtt2\" (UID: \"62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d\") " pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:29.876402 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:29.876287    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 19:56:29.876402 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:29.876312    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 19:56:29.876402 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:29.876326    2580 projected.go:196] Error preparing data for projected volume kube-api-access-tdmfr for pod openshift-network-diagnostics/network-check-target-5gtt2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 19:56:29.876402 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:29.876384    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr podName:62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d nodeName:}" failed. No retries permitted until 2026-07-20 19:56:33.876364864 +0000 UTC m=+9.140434895 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdmfr" (UniqueName: "kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr") pod "network-check-target-5gtt2" (UID: "62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 19:56:30.170814 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:30.170728    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:30.170999 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:30.170893    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5778m" podUID="c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1"
Jul 20 19:56:30.170999 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:30.170923    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:30.171129 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:30.171053    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xlc4n" podUID="2fbbdeaa-171d-4032-9bd3-7bbd4c009388"
Jul 20 19:56:30.171129 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:30.171097    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:30.171234 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:30.171195    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gtt2" podUID="62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d"
Jul 20 19:56:32.170341 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:32.170305    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:32.170880 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:32.170305    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:32.170880 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:32.170305    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:32.170880 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:32.170470    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gtt2" podUID="62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d"
Jul 20 19:56:32.170880 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:32.170527    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5778m" podUID="c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1"
Jul 20 19:56:32.170880 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:32.170645    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xlc4n" podUID="2fbbdeaa-171d-4032-9bd3-7bbd4c009388"
Jul 20 19:56:33.710615 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:33.710271    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret\") pod \"global-pull-secret-syncer-xlc4n\" (UID: \"2fbbdeaa-171d-4032-9bd3-7bbd4c009388\") " pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:33.710615 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:33.710326    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs\") pod \"network-metrics-daemon-5778m\" (UID: \"c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1\") " pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:33.710615 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:33.710452    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 19:56:33.710615 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:33.710492    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 19:56:33.710615 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:33.710509    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs podName:c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1 nodeName:}" failed. No retries permitted until 2026-07-20 19:56:41.710491313 +0000 UTC m=+16.974561340 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs") pod "network-metrics-daemon-5778m" (UID: "c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 19:56:33.710615 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:33.710555    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret podName:2fbbdeaa-171d-4032-9bd3-7bbd4c009388 nodeName:}" failed. No retries permitted until 2026-07-20 19:56:41.710535638 +0000 UTC m=+16.974605678 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret") pod "global-pull-secret-syncer-xlc4n" (UID: "2fbbdeaa-171d-4032-9bd3-7bbd4c009388") : object "kube-system"/"original-pull-secret" not registered
Jul 20 19:56:33.912009 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:33.911971    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdmfr\" (UniqueName: \"kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr\") pod \"network-check-target-5gtt2\" (UID: \"62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d\") " pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:33.912195 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:33.912085    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 19:56:33.912195 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:33.912105    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 19:56:33.912195 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:33.912116    2580 projected.go:196] Error preparing data for projected volume kube-api-access-tdmfr for pod openshift-network-diagnostics/network-check-target-5gtt2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 19:56:33.912195 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:33.912179    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr podName:62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d nodeName:}" failed. No retries permitted until 2026-07-20 19:56:41.91215958 +0000 UTC m=+17.176229608 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdmfr" (UniqueName: "kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr") pod "network-check-target-5gtt2" (UID: "62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 19:56:34.171577 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:34.170697    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:34.171577 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:34.170850    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xlc4n" podUID="2fbbdeaa-171d-4032-9bd3-7bbd4c009388"
Jul 20 19:56:34.171577 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:34.171303    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:34.171577 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:34.171397    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gtt2" podUID="62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d"
Jul 20 19:56:34.171577 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:34.171443    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:34.171577 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:34.171529    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5778m" podUID="c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1"
Jul 20 19:56:36.170525 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:36.170494    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:36.170984 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:36.170590    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:36.170984 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:36.170596    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gtt2" podUID="62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d"
Jul 20 19:56:36.170984 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:36.170619    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:36.170984 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:36.170720    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5778m" podUID="c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1"
Jul 20 19:56:36.170984 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:36.170776    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xlc4n" podUID="2fbbdeaa-171d-4032-9bd3-7bbd4c009388"
Jul 20 19:56:38.170481 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:38.170363    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:38.170481 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:38.170412    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:38.170481 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:38.170413    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:38.171066 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:38.170496    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gtt2" podUID="62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d"
Jul 20 19:56:38.171066 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:38.170584    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xlc4n" podUID="2fbbdeaa-171d-4032-9bd3-7bbd4c009388"
Jul 20 19:56:38.171066 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:38.170667    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5778m" podUID="c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1"
Jul 20 19:56:40.170403 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:40.170364    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:40.170403 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:40.170406    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:40.170921 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:40.170413    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:40.170921 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:40.170506    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xlc4n" podUID="2fbbdeaa-171d-4032-9bd3-7bbd4c009388"
Jul 20 19:56:40.170921 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:40.170620    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5778m" podUID="c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1"
Jul 20 19:56:40.170921 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:40.170691    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gtt2" podUID="62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d"
Jul 20 19:56:41.770518 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:41.770483    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret\") pod \"global-pull-secret-syncer-xlc4n\" (UID: \"2fbbdeaa-171d-4032-9bd3-7bbd4c009388\") " pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:41.770518 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:41.770533    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs\") pod \"network-metrics-daemon-5778m\" (UID: \"c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1\") " pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:41.771026 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:41.770635    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 19:56:41.771026 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:41.770638    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 19:56:41.771026 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:41.770704    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs podName:c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1 nodeName:}" failed. No retries permitted until 2026-07-20 19:56:57.770683543 +0000 UTC m=+33.034753568 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs") pod "network-metrics-daemon-5778m" (UID: "c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 19:56:41.771026 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:41.770723    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret podName:2fbbdeaa-171d-4032-9bd3-7bbd4c009388 nodeName:}" failed. No retries permitted until 2026-07-20 19:56:57.770714534 +0000 UTC m=+33.034784558 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret") pod "global-pull-secret-syncer-xlc4n" (UID: "2fbbdeaa-171d-4032-9bd3-7bbd4c009388") : object "kube-system"/"original-pull-secret" not registered
Jul 20 19:56:41.972533 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:41.972429    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdmfr\" (UniqueName: \"kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr\") pod \"network-check-target-5gtt2\" (UID: \"62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d\") " pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:41.972697 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:41.972588    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 19:56:41.972697 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:41.972609    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 19:56:41.972697 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:41.972623    2580 projected.go:196] Error preparing data for projected volume kube-api-access-tdmfr for pod openshift-network-diagnostics/network-check-target-5gtt2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 19:56:41.972697 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:41.972683    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr podName:62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d nodeName:}" failed. No retries permitted until 2026-07-20 19:56:57.972664863 +0000 UTC m=+33.236734909 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdmfr" (UniqueName: "kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr") pod "network-check-target-5gtt2" (UID: "62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 19:56:42.170298 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:42.170265    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:42.170540 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:42.170367    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:42.170540 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:42.170372    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gtt2" podUID="62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d"
Jul 20 19:56:42.170540 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:42.170487    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xlc4n" podUID="2fbbdeaa-171d-4032-9bd3-7bbd4c009388"
Jul 20 19:56:42.170708 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:42.170552    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:42.170708 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:42.170640    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5778m" podUID="c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1"
Jul 20 19:56:44.170450 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:44.170420    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:44.170899 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:44.170420    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:44.170899 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:44.170540    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gtt2" podUID="62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d"
Jul 20 19:56:44.170899 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:44.170420    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:44.170899 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:44.170610    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5778m" podUID="c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1"
Jul 20 19:56:44.170899 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:44.170682    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xlc4n" podUID="2fbbdeaa-171d-4032-9bd3-7bbd4c009388"
Jul 20 19:56:46.170785 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:46.170630    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:46.171135 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:46.170630    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:46.171135 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:46.170874    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5778m" podUID="c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1"
Jul 20 19:56:46.171135 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:46.170630    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:46.171135 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:46.170933    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gtt2" podUID="62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d"
Jul 20 19:56:46.171135 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:46.171016    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xlc4n" podUID="2fbbdeaa-171d-4032-9bd3-7bbd4c009388"
Jul 20 19:56:46.433360 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:46.433341    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 19:56:46.433788 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:46.433761    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a0e6c85-e495-4711-a23e-acdb2c031587" containerID="89e801fe95240e18b3272403431b6ca0757b380e6a1b6af3c0731e8a0abb2bde" exitCode=1
Jul 20 19:56:46.433876 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:46.433835    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4" event={"ID":"1a0e6c85-e495-4711-a23e-acdb2c031587","Type":"ContainerStarted","Data":"2efae8a3e2d70b9b4ef7e44084d4a70ff3929947e6566a31bdb73e6e8993269b"}
Jul 20 19:56:46.433943 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:46.433927    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4" event={"ID":"1a0e6c85-e495-4711-a23e-acdb2c031587","Type":"ContainerDied","Data":"89e801fe95240e18b3272403431b6ca0757b380e6a1b6af3c0731e8a0abb2bde"}
Jul 20 19:56:46.434032 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:46.434000    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4" event={"ID":"1a0e6c85-e495-4711-a23e-acdb2c031587","Type":"ContainerStarted","Data":"42314d3835e10736da6edbf9151905f9c05a03ac9827289112bfdf8184cab08a"}
Jul 20 19:56:46.435477 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:46.435451    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jzsk9" event={"ID":"eddfe126-5be8-4f75-80bc-97f01e169ba4","Type":"ContainerStarted","Data":"a349d1f7a22afd703e15e32ea755d434bea8c6da17f9a38d93e9dcffb71b94ad"}
Jul 20 19:56:46.436972 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:46.436938    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-191.ec2.internal" event={"ID":"504046be7821a23dec7f8227b49cfc17","Type":"ContainerStarted","Data":"16fc611a964031d4485927f21a53909a2efe71189b3e827ac3aacfb3f46b913f"}
Jul 20 19:56:46.438495 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:46.438475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v4zzw" event={"ID":"d2b375ee-5209-4c9d-9ae8-baa440fb4f03","Type":"ContainerStarted","Data":"3b8980d2215b15de109393999c0501781189f4a6184d3341f39df353927d3ff1"}
Jul 20 19:56:46.471840 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:46.471791    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jzsk9" podStartSLOduration=1.618601398 podStartE2EDuration="21.471774816s" totalImagesPullingTime="19.853173418s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:56:46.453639814 +0000 UTC m=+21.717709861" watchObservedRunningTime="2026-07-20 19:56:46.471774816 +0000 UTC m=+21.735844861"
Jul 20 19:56:46.472019 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:46.471998    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-v4zzw" podStartSLOduration=1.877724242 podStartE2EDuration="21.471992079s" totalImagesPullingTime="19.594267837s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:56:46.471441631 +0000 UTC m=+21.735511680" watchObservedRunningTime="2026-07-20 19:56:46.471992079 +0000 UTC m=+21.736062125"
Jul 20 19:56:47.442153 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:47.441931    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-59kk4" event={"ID":"8f2f4dc6-3966-4eff-aecc-06298645ecb5","Type":"ContainerStarted","Data":"cffd061d524ddaf79daf57353341ff30251e82e695abda0361511fa6181f8df0"}
Jul 20 19:56:47.443494 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:47.443466    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nb8cs" event={"ID":"d518f591-565e-4ac1-b5d1-f88311868032","Type":"ContainerStarted","Data":"999a15da7523b674792952515b12f4a619fc7fc17648ad52f410e8ece0a5ab07"}
Jul 20 19:56:47.446492 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:47.446469    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 19:56:47.446869 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:47.446847    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4" event={"ID":"1a0e6c85-e495-4711-a23e-acdb2c031587","Type":"ContainerStarted","Data":"17ae1e2a056d06733cb971ce2089acf56eeccc407b94893cfbfda508a4e4f360"}
Jul 20 19:56:47.447062 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:47.446925    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4" event={"ID":"1a0e6c85-e495-4711-a23e-acdb2c031587","Type":"ContainerStarted","Data":"854192a1f3059e953f0f3f4432363e9efe7e8a585d89a6f9976a1692afe22ffd"}
Jul 20 19:56:47.447062 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:47.446993    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4" event={"ID":"1a0e6c85-e495-4711-a23e-acdb2c031587","Type":"ContainerStarted","Data":"fd0ca4d1187e077838550639f376baf508254ae146760cc17238195ad4973b4e"}
Jul 20 19:56:47.448375 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:47.448350    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nngvn" event={"ID":"a164b5a7-e049-42a5-8f89-915704f82f68","Type":"ContainerStarted","Data":"cee22e4f5cba208950e010d7eb860dfbc48e0332550a4f01edd173fd160297b3"}
Jul 20 19:56:47.450017 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:47.449991    2580 generic.go:356] "Generic (PLEG): container finished" podID="bfaea5fd-9480-4cd0-b133-217c1eb6069a" containerID="df1442dac9012e21c127e0850907f99cfbd17d2169cd6a85d5200920651dd78b" exitCode=0
Jul 20 19:56:47.450096 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:47.450061    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b94k5" event={"ID":"bfaea5fd-9480-4cd0-b133-217c1eb6069a","Type":"ContainerDied","Data":"df1442dac9012e21c127e0850907f99cfbd17d2169cd6a85d5200920651dd78b"}
Jul 20 19:56:47.451638 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:47.451617    2580 generic.go:356] "Generic (PLEG): container finished" podID="0c5e8c7baca191c444a3231e6958085d" containerID="c5c145fd825ec15682b072d26a5b0c2dd5b1cf6f0ecb9e1f2f0c1bc8b63a5803" exitCode=0
Jul 20 19:56:47.451771 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:47.451700    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-191.ec2.internal" event={"ID":"0c5e8c7baca191c444a3231e6958085d","Type":"ContainerDied","Data":"c5c145fd825ec15682b072d26a5b0c2dd5b1cf6f0ecb9e1f2f0c1bc8b63a5803"}
Jul 20 19:56:47.453361 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:47.453327    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w5555" event={"ID":"b4c1d9fe-73ea-4091-9855-b9f6970d176e","Type":"ContainerStarted","Data":"30faebc0da09f8871a345e2ddf9a49988eb0a391508577ae9ce8ee1fc969ed2b"}
Jul 20 19:56:47.454885 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:47.454860    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf" event={"ID":"36752684-c032-4e36-bbe5-65480fc3d8d8","Type":"ContainerStarted","Data":"1f7264f7ed7a027dfba3b60b26e524853b224a3d71b3b09f42a83369f89a864e"}
Jul 20 19:56:47.457032 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:47.456995    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-191.ec2.internal" podStartSLOduration=22.456914575 podStartE2EDuration="22.456914575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:56:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:56:46.488293049 +0000 UTC m=+21.752363096" watchObservedRunningTime="2026-07-20 19:56:47.456914575 +0000 UTC m=+22.720984622"
Jul 20 19:56:47.457349 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:47.457326    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-59kk4" podStartSLOduration=3.041917881 podStartE2EDuration="22.457318551s" totalImagesPullingTime="19.41540067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:56:47.456678408 +0000 UTC m=+22.720748455" watchObservedRunningTime="2026-07-20 19:56:47.457318551 +0000 UTC m=+22.721388597"
Jul 20 19:56:47.493885 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:47.493825    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-nngvn" podStartSLOduration=7.244220428 podStartE2EDuration="22.493806409s" totalImagesPullingTime="15.249585981s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:56:47.475109319 +0000 UTC m=+22.739179366" watchObservedRunningTime="2026-07-20 19:56:47.493806409 +0000 UTC m=+22.757876458"
Jul 20 19:56:47.527885 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:47.527835    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nb8cs" podStartSLOduration=7.303648869 podStartE2EDuration="22.527817299s" totalImagesPullingTime="15.22416843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:56:47.527352116 +0000 UTC m=+22.791422164" watchObservedRunningTime="2026-07-20 19:56:47.527817299 +0000 UTC m=+22.791887347"
Jul 20 19:56:47.545345 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:47.545235    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-w5555" podStartSLOduration=3.035465215 podStartE2EDuration="22.545207581s" totalImagesPullingTime="19.509742366s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:56:47.544950977 +0000 UTC m=+22.809021023" watchObservedRunningTime="2026-07-20 19:56:47.545207581 +0000 UTC m=+22.809277629"
Jul 20 19:56:47.922215 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:47.922193    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 20 19:56:48.097734 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:48.097576    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-20T19:56:47.922212393Z","UUID":"5acdf3cc-0945-4b91-89c6-e880c555e7e5","Handler":null,"Name":"","Endpoint":""}
Jul 20 19:56:48.100845 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:48.100813    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 20 19:56:48.100984 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:48.100861    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 20 19:56:48.170021 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:48.169987    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:48.170021 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:48.170020    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:48.170282 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:48.170034    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:48.170282 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:48.170119    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xlc4n" podUID="2fbbdeaa-171d-4032-9bd3-7bbd4c009388"
Jul 20 19:56:48.170282 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:48.170178    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gtt2" podUID="62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d"
Jul 20 19:56:48.170282 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:48.170255    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5778m" podUID="c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1"
Jul 20 19:56:48.459921 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:48.459841    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-191.ec2.internal" event={"ID":"0c5e8c7baca191c444a3231e6958085d","Type":"ContainerStarted","Data":"b785d0f7a51c7181a4bad254a99822d8835ea85629e38aa2032c1757762412d3"}
Jul 20 19:56:48.462904 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:48.462644    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf" event={"ID":"36752684-c032-4e36-bbe5-65480fc3d8d8","Type":"ContainerStarted","Data":"37077d962703505f33ec83980446ffcfc2e6fb4aad591874e3433a7c79b0336f"}
Jul 20 19:56:49.467725 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:49.467579    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf" event={"ID":"36752684-c032-4e36-bbe5-65480fc3d8d8","Type":"ContainerStarted","Data":"ba2e8938c2bd641c8438a0a1f6b6612971c8e6c8b72e5f87ea283f1bd416a469"}
Jul 20 19:56:49.471538 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:49.471510    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 19:56:49.471991 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:49.471951    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4" event={"ID":"1a0e6c85-e495-4711-a23e-acdb2c031587","Type":"ContainerStarted","Data":"47b842022dbf45a96c349eff537d4dd35051e034f23dcd472bf89fccad3dd2d8"}
Jul 20 19:56:49.487217 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:49.487172    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7jf" podStartSLOduration=2.465173166 podStartE2EDuration="24.487157245s" totalImagesPullingTime="22.021984079s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:56:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:56:49.486880632 +0000 UTC m=+24.750950677" watchObservedRunningTime="2026-07-20 19:56:49.487157245 +0000 UTC m=+24.751227292"
Jul 20 19:56:49.487664 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:49.487634    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-191.ec2.internal" podStartSLOduration=24.487625046 podStartE2EDuration="24.487625046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:56:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:56:48.475679999 +0000 UTC m=+23.739750046" watchObservedRunningTime="2026-07-20 19:56:49.487625046 +0000 UTC m=+24.751695093"
Jul 20 19:56:49.756742 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:49.756662    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nb8cs"
Jul 20 19:56:49.757385 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:49.757361    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nb8cs"
Jul 20 19:56:50.170095 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:50.170058    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:50.170095 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:50.170093    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:50.170432 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:50.170058    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:50.170432 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:50.170174    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5778m" podUID="c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1"
Jul 20 19:56:50.170432 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:50.170263    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xlc4n" podUID="2fbbdeaa-171d-4032-9bd3-7bbd4c009388"
Jul 20 19:56:50.170432 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:50.170347    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gtt2" podUID="62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d"
Jul 20 19:56:50.474399 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:50.474316    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nb8cs"
Jul 20 19:56:50.474823 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:50.474537    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nb8cs"
Jul 20 19:56:50.496602 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:50.496569    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nb8cs_d518f591-565e-4ac1-b5d1-f88311868032/konnectivity-agent/0.log" limit=1024
Jul 20 19:56:50.577943 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:50.577903    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nb8cs_d518f591-565e-4ac1-b5d1-f88311868032/konnectivity-agent/0.log" limit=1024
Jul 20 19:56:52.170054 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:52.169883    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:52.170503 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:52.169885    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:52.170503 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:52.170159    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gtt2" podUID="62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d"
Jul 20 19:56:52.170503 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:52.169885    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:52.170503 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:52.170221    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xlc4n" podUID="2fbbdeaa-171d-4032-9bd3-7bbd4c009388"
Jul 20 19:56:52.170503 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:52.170316    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5778m" podUID="c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1"
Jul 20 19:56:52.482061 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:52.481988    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 19:56:52.482413 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:52.482384    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4" event={"ID":"1a0e6c85-e495-4711-a23e-acdb2c031587","Type":"ContainerStarted","Data":"20c20c07ddea18717821b97a33965363992adf8e0ccb2e7f775d28bbc1d75f53"}
Jul 20 19:56:52.482652 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:52.482626    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:52.482763 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:52.482695    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:52.482825 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:52.482808    2580 scope.go:117] "RemoveContainer" containerID="89e801fe95240e18b3272403431b6ca0757b380e6a1b6af3c0731e8a0abb2bde"
Jul 20 19:56:52.484319 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:52.484294    2580 generic.go:356] "Generic (PLEG): container finished" podID="bfaea5fd-9480-4cd0-b133-217c1eb6069a" containerID="62af6eac33d10bb7f827e47de5792616dedf493e0252f420b095b75539ee06aa" exitCode=0
Jul 20 19:56:52.484401 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:52.484333    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b94k5" event={"ID":"bfaea5fd-9480-4cd0-b133-217c1eb6069a","Type":"ContainerDied","Data":"62af6eac33d10bb7f827e47de5792616dedf493e0252f420b095b75539ee06aa"}
Jul 20 19:56:52.499647 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:52.499627    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:53.492360 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:53.492283    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 19:56:53.493190 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:53.492709    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4" event={"ID":"1a0e6c85-e495-4711-a23e-acdb2c031587","Type":"ContainerStarted","Data":"7e9252879a172f25ce633643328a13b50e46f4449b10117fd6f388ec95a2734e"}
Jul 20 19:56:53.493190 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:53.493025    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:53.507776 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:53.507754    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:56:53.524729 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:53.524682    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4" podStartSLOduration=8.91640109 podStartE2EDuration="28.524669156s" totalImagesPullingTime="19.608268066s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:56:53.524460944 +0000 UTC m=+28.788531000" watchObservedRunningTime="2026-07-20 19:56:53.524669156 +0000 UTC m=+28.788739203"
Jul 20 19:56:53.866095 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:53.865881    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5778m"]
Jul 20 19:56:53.866263 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:53.866197    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:53.866312 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:53.866285    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5778m" podUID="c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1"
Jul 20 19:56:53.869593 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:53.869568    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xlc4n"]
Jul 20 19:56:53.869698 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:53.869686    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:53.869785 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:53.869766    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xlc4n" podUID="2fbbdeaa-171d-4032-9bd3-7bbd4c009388"
Jul 20 19:56:53.870121 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:53.870101    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5gtt2"]
Jul 20 19:56:53.870207 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:53.870196    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:53.870290 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:53.870272    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gtt2" podUID="62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d"
Jul 20 19:56:54.136331 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:54.136242    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nb8cs_d518f591-565e-4ac1-b5d1-f88311868032/konnectivity-agent/0.log" limit=1024
Jul 20 19:56:54.232779 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:54.232740    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nb8cs_d518f591-565e-4ac1-b5d1-f88311868032/konnectivity-agent/0.log" limit=1024
Jul 20 19:56:54.497428 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:54.497347    2580 generic.go:356] "Generic (PLEG): container finished" podID="bfaea5fd-9480-4cd0-b133-217c1eb6069a" containerID="dc8895143321409f166d91525db25d7b67d48a2a76b4fe9b01eabb7c8bb8ca59" exitCode=0
Jul 20 19:56:54.497428 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:54.497408    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b94k5" event={"ID":"bfaea5fd-9480-4cd0-b133-217c1eb6069a","Type":"ContainerDied","Data":"dc8895143321409f166d91525db25d7b67d48a2a76b4fe9b01eabb7c8bb8ca59"}
Jul 20 19:56:55.171361 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:55.171322    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:55.171541 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:55.171419    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:55.171541 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:55.171455    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:55.171658 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:55.171535    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gtt2" podUID="62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d"
Jul 20 19:56:55.171658 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:55.171525    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5778m" podUID="c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1"
Jul 20 19:56:55.171658 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:55.171614    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xlc4n" podUID="2fbbdeaa-171d-4032-9bd3-7bbd4c009388"
Jul 20 19:56:56.504210 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:56.504174    2580 generic.go:356] "Generic (PLEG): container finished" podID="bfaea5fd-9480-4cd0-b133-217c1eb6069a" containerID="470ead24ed6d0050b361e48e41f0fadeadfd5c8e97df66230d0860869c977c7e" exitCode=0
Jul 20 19:56:56.504586 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:56.504225    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b94k5" event={"ID":"bfaea5fd-9480-4cd0-b133-217c1eb6069a","Type":"ContainerDied","Data":"470ead24ed6d0050b361e48e41f0fadeadfd5c8e97df66230d0860869c977c7e"}
Jul 20 19:56:57.170744 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:57.170709    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:57.170914 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:57.170710    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:57.170914 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:57.170843    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5778m" podUID="c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1"
Jul 20 19:56:57.171082 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:57.170918    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xlc4n" podUID="2fbbdeaa-171d-4032-9bd3-7bbd4c009388"
Jul 20 19:56:57.171082 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:57.170715    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:57.171082 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:57.171043    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gtt2" podUID="62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d"
Jul 20 19:56:57.794483 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:57.794401    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret\") pod \"global-pull-secret-syncer-xlc4n\" (UID: \"2fbbdeaa-171d-4032-9bd3-7bbd4c009388\") " pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:57.794483 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:57.794442    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs\") pod \"network-metrics-daemon-5778m\" (UID: \"c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1\") " pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:57.794920 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:57.794551    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 19:56:57.794920 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:57.794596    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs podName:c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1 nodeName:}" failed. No retries permitted until 2026-07-20 19:57:29.794582574 +0000 UTC m=+65.058652598 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs") pod "network-metrics-daemon-5778m" (UID: "c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 19:56:57.794920 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:57.794552    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 19:56:57.794920 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:57.794685    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret podName:2fbbdeaa-171d-4032-9bd3-7bbd4c009388 nodeName:}" failed. No retries permitted until 2026-07-20 19:57:29.794669024 +0000 UTC m=+65.058739070 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret") pod "global-pull-secret-syncer-xlc4n" (UID: "2fbbdeaa-171d-4032-9bd3-7bbd4c009388") : object "kube-system"/"original-pull-secret" not registered
Jul 20 19:56:57.995426 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:57.995387    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdmfr\" (UniqueName: \"kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr\") pod \"network-check-target-5gtt2\" (UID: \"62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d\") " pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:57.995621 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:57.995570    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 19:56:57.995621 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:57.995598    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 19:56:57.995621 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:57.995614    2580 projected.go:196] Error preparing data for projected volume kube-api-access-tdmfr for pod openshift-network-diagnostics/network-check-target-5gtt2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 19:56:57.995741 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:57.995679    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr podName:62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d nodeName:}" failed. No retries permitted until 2026-07-20 19:57:29.995660655 +0000 UTC m=+65.259730681 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdmfr" (UniqueName: "kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr") pod "network-check-target-5gtt2" (UID: "62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 19:56:59.102053 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.102019    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-191.ec2.internal" event="NodeReady"
Jul 20 19:56:59.102497 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.102143    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 20 19:56:59.143579 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.143545    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"]
Jul 20 19:56:59.147142 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.147117    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.150645 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.150617    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xzmhf\""
Jul 20 19:56:59.150892 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.150867    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 20 19:56:59.151039 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.150894    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 20 19:56:59.151039 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.150928    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 20 19:56:59.157440 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.157416    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nn6k7"]
Jul 20 19:56:59.157572 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.157555    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 20 19:56:59.161208 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.161189    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nn6k7"
Jul 20 19:56:59.164175 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.164158    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"]
Jul 20 19:56:59.164933 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.164911    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xvdd8"]
Jul 20 19:56:59.165410 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.165383    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 20 19:56:59.165670 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.165645    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 20 19:56:59.165804 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.165786    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fzgwk\""
Jul 20 19:56:59.168267 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.168246    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xvdd8"
Jul 20 19:56:59.175536 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.174415    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qsckw\""
Jul 20 19:56:59.175536 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.174700    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 20 19:56:59.175536 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.174861    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 20 19:56:59.175536 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.174942    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 20 19:56:59.178532 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.178512    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nn6k7"]
Jul 20 19:56:59.179385 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.178665    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:56:59.179385 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.178978    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:56:59.179385 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.179303    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:56:59.182294 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.182277    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 20 19:56:59.182584 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.182356    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 20 19:56:59.183002 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.182952    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 20 19:56:59.183103 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.183048    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 20 19:56:59.183541 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.183525    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mmqwm\""
Jul 20 19:56:59.183664 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.183646    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qx848\""
Jul 20 19:56:59.184448 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.184431    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xvdd8"]
Jul 20 19:56:59.304345 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.304307    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ac271c1-f11b-4518-a768-697ed858df99-cert\") pod \"ingress-canary-xvdd8\" (UID: \"9ac271c1-f11b-4518-a768-697ed858df99\") " pod="openshift-ingress-canary/ingress-canary-xvdd8"
Jul 20 19:56:59.304497 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.304354    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-tls\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.304497 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.304422    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b9c6f82-5df3-4fec-b041-a6f3615d7949-metrics-tls\") pod \"dns-default-nn6k7\" (UID: \"3b9c6f82-5df3-4fec-b041-a6f3615d7949\") " pod="openshift-dns/dns-default-nn6k7"
Jul 20 19:56:59.304497 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.304454    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3b9c6f82-5df3-4fec-b041-a6f3615d7949-config-volume\") pod \"dns-default-nn6k7\" (UID: \"3b9c6f82-5df3-4fec-b041-a6f3615d7949\") " pod="openshift-dns/dns-default-nn6k7"
Jul 20 19:56:59.304642 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.304519    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e441b5f3-c7c5-4981-bc51-c797d2b43002-image-registry-private-configuration\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.304642 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.304550    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e441b5f3-c7c5-4981-bc51-c797d2b43002-trusted-ca\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.304642 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.304577    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvn4b\" (UniqueName: \"kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-kube-api-access-jvn4b\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.304642 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.304622    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlqrc\" (UniqueName: \"kubernetes.io/projected/9ac271c1-f11b-4518-a768-697ed858df99-kube-api-access-dlqrc\") pod \"ingress-canary-xvdd8\" (UID: \"9ac271c1-f11b-4518-a768-697ed858df99\") " pod="openshift-ingress-canary/ingress-canary-xvdd8"
Jul 20 19:56:59.304841 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.304685    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9bn9\" (UniqueName: \"kubernetes.io/projected/3b9c6f82-5df3-4fec-b041-a6f3615d7949-kube-api-access-l9bn9\") pod \"dns-default-nn6k7\" (UID: \"3b9c6f82-5df3-4fec-b041-a6f3615d7949\") " pod="openshift-dns/dns-default-nn6k7"
Jul 20 19:56:59.304841 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.304753    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-bound-sa-token\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.304841 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.304779    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e441b5f3-c7c5-4981-bc51-c797d2b43002-installation-pull-secrets\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.304841 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.304832    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e441b5f3-c7c5-4981-bc51-c797d2b43002-ca-trust-extracted\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.305048 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.304883    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b9c6f82-5df3-4fec-b041-a6f3615d7949-tmp-dir\") pod \"dns-default-nn6k7\" (UID: \"3b9c6f82-5df3-4fec-b041-a6f3615d7949\") " pod="openshift-dns/dns-default-nn6k7"
Jul 20 19:56:59.305048 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.304912    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-certificates\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.406201 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.406114    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b9c6f82-5df3-4fec-b041-a6f3615d7949-metrics-tls\") pod \"dns-default-nn6k7\" (UID: \"3b9c6f82-5df3-4fec-b041-a6f3615d7949\") " pod="openshift-dns/dns-default-nn6k7"
Jul 20 19:56:59.406201 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.406173    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3b9c6f82-5df3-4fec-b041-a6f3615d7949-config-volume\") pod \"dns-default-nn6k7\" (UID: \"3b9c6f82-5df3-4fec-b041-a6f3615d7949\") " pod="openshift-dns/dns-default-nn6k7"
Jul 20 19:56:59.406201 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.406202    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e441b5f3-c7c5-4981-bc51-c797d2b43002-image-registry-private-configuration\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.406477 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.406220    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e441b5f3-c7c5-4981-bc51-c797d2b43002-trusted-ca\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.406477 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.406236    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvn4b\" (UniqueName: \"kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-kube-api-access-jvn4b\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.406477 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.406255    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlqrc\" (UniqueName: \"kubernetes.io/projected/9ac271c1-f11b-4518-a768-697ed858df99-kube-api-access-dlqrc\") pod \"ingress-canary-xvdd8\" (UID: \"9ac271c1-f11b-4518-a768-697ed858df99\") " pod="openshift-ingress-canary/ingress-canary-xvdd8"
Jul 20 19:56:59.406477 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:59.406273    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 19:56:59.406477 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.406283    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9bn9\" (UniqueName: \"kubernetes.io/projected/3b9c6f82-5df3-4fec-b041-a6f3615d7949-kube-api-access-l9bn9\") pod \"dns-default-nn6k7\" (UID: \"3b9c6f82-5df3-4fec-b041-a6f3615d7949\") " pod="openshift-dns/dns-default-nn6k7"
Jul 20 19:56:59.406477 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.406318    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-bound-sa-token\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.406477 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:59.406343    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b9c6f82-5df3-4fec-b041-a6f3615d7949-metrics-tls podName:3b9c6f82-5df3-4fec-b041-a6f3615d7949 nodeName:}" failed. No retries permitted until 2026-07-20 19:56:59.906322312 +0000 UTC m=+35.170392336 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b9c6f82-5df3-4fec-b041-a6f3615d7949-metrics-tls") pod "dns-default-nn6k7" (UID: "3b9c6f82-5df3-4fec-b041-a6f3615d7949") : secret "dns-default-metrics-tls" not found
Jul 20 19:56:59.406477 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.406380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e441b5f3-c7c5-4981-bc51-c797d2b43002-installation-pull-secrets\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.406477 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.406428    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e441b5f3-c7c5-4981-bc51-c797d2b43002-ca-trust-extracted\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.406477 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.406459    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b9c6f82-5df3-4fec-b041-a6f3615d7949-tmp-dir\") pod \"dns-default-nn6k7\" (UID: \"3b9c6f82-5df3-4fec-b041-a6f3615d7949\") " pod="openshift-dns/dns-default-nn6k7"
Jul 20 19:56:59.407001 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.406484    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-certificates\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.407001 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.406516    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ac271c1-f11b-4518-a768-697ed858df99-cert\") pod \"ingress-canary-xvdd8\" (UID: \"9ac271c1-f11b-4518-a768-697ed858df99\") " pod="openshift-ingress-canary/ingress-canary-xvdd8"
Jul 20 19:56:59.407001 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.406541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-tls\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.407001 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.406876    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3b9c6f82-5df3-4fec-b041-a6f3615d7949-config-volume\") pod \"dns-default-nn6k7\" (UID: \"3b9c6f82-5df3-4fec-b041-a6f3615d7949\") " pod="openshift-dns/dns-default-nn6k7"
Jul 20 19:56:59.407001 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.406903    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e441b5f3-c7c5-4981-bc51-c797d2b43002-ca-trust-extracted\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.407244 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:59.407012    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 19:56:59.407244 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:59.407026    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b44dfd7b9-5lwkw: secret "image-registry-tls" not found
Jul 20 19:56:59.407244 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:59.407071    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-tls podName:e441b5f3-c7c5-4981-bc51-c797d2b43002 nodeName:}" failed. No retries permitted until 2026-07-20 19:56:59.907055573 +0000 UTC m=+35.171125600 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-tls") pod "image-registry-5b44dfd7b9-5lwkw" (UID: "e441b5f3-c7c5-4981-bc51-c797d2b43002") : secret "image-registry-tls" not found
Jul 20 19:56:59.407390 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.407376    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-certificates\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.407770 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.407612    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b9c6f82-5df3-4fec-b041-a6f3615d7949-tmp-dir\") pod \"dns-default-nn6k7\" (UID: \"3b9c6f82-5df3-4fec-b041-a6f3615d7949\") " pod="openshift-dns/dns-default-nn6k7"
Jul 20 19:56:59.408080 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.408059    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e441b5f3-c7c5-4981-bc51-c797d2b43002-trusted-ca\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.411386 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.411360    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e441b5f3-c7c5-4981-bc51-c797d2b43002-installation-pull-secrets\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.411498 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.411360    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e441b5f3-c7c5-4981-bc51-c797d2b43002-image-registry-private-configuration\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.411498 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.411398    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ac271c1-f11b-4518-a768-697ed858df99-cert\") pod \"ingress-canary-xvdd8\" (UID: \"9ac271c1-f11b-4518-a768-697ed858df99\") " pod="openshift-ingress-canary/ingress-canary-xvdd8"
Jul 20 19:56:59.415047 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.415024    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-bound-sa-token\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.415158 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.415130    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvn4b\" (UniqueName: \"kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-kube-api-access-jvn4b\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.415931 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.415910    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlqrc\" (UniqueName: \"kubernetes.io/projected/9ac271c1-f11b-4518-a768-697ed858df99-kube-api-access-dlqrc\") pod \"ingress-canary-xvdd8\" (UID: \"9ac271c1-f11b-4518-a768-697ed858df99\") " pod="openshift-ingress-canary/ingress-canary-xvdd8"
Jul 20 19:56:59.417404 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.417374    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9bn9\" (UniqueName: \"kubernetes.io/projected/3b9c6f82-5df3-4fec-b041-a6f3615d7949-kube-api-access-l9bn9\") pod \"dns-default-nn6k7\" (UID: \"3b9c6f82-5df3-4fec-b041-a6f3615d7949\") " pod="openshift-dns/dns-default-nn6k7"
Jul 20 19:56:59.482235 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.482199    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xvdd8"
Jul 20 19:56:59.912678 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.912641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b9c6f82-5df3-4fec-b041-a6f3615d7949-metrics-tls\") pod \"dns-default-nn6k7\" (UID: \"3b9c6f82-5df3-4fec-b041-a6f3615d7949\") " pod="openshift-dns/dns-default-nn6k7"
Jul 20 19:56:59.912868 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:56:59.912758    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-tls\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:56:59.912868 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:59.912814    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 19:56:59.913022 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:59.912899    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b9c6f82-5df3-4fec-b041-a6f3615d7949-metrics-tls podName:3b9c6f82-5df3-4fec-b041-a6f3615d7949 nodeName:}" failed. No retries permitted until 2026-07-20 19:57:00.912878707 +0000 UTC m=+36.176948738 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b9c6f82-5df3-4fec-b041-a6f3615d7949-metrics-tls") pod "dns-default-nn6k7" (UID: "3b9c6f82-5df3-4fec-b041-a6f3615d7949") : secret "dns-default-metrics-tls" not found
Jul 20 19:56:59.913022 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:59.912907    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 19:56:59.913022 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:59.912923    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b44dfd7b9-5lwkw: secret "image-registry-tls" not found
Jul 20 19:56:59.913022 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:56:59.912992    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-tls podName:e441b5f3-c7c5-4981-bc51-c797d2b43002 nodeName:}" failed. No retries permitted until 2026-07-20 19:57:00.912975729 +0000 UTC m=+36.177045755 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-tls") pod "image-registry-5b44dfd7b9-5lwkw" (UID: "e441b5f3-c7c5-4981-bc51-c797d2b43002") : secret "image-registry-tls" not found
Jul 20 19:57:00.749465 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:00.749439    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xvdd8"]
Jul 20 19:57:00.752847 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:57:00.752812    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ac271c1_f11b_4518_a768_697ed858df99.slice/crio-6b4a9fa9fae0c2883d2af7c8c7399135ed50abcbcf973a4c7b06c5d9cfa76a61 WatchSource:0}: Error finding container 6b4a9fa9fae0c2883d2af7c8c7399135ed50abcbcf973a4c7b06c5d9cfa76a61: Status 404 returned error can't find the container with id 6b4a9fa9fae0c2883d2af7c8c7399135ed50abcbcf973a4c7b06c5d9cfa76a61
Jul 20 19:57:00.921895 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:00.921867    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-tls\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:57:00.922032 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:00.921916    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b9c6f82-5df3-4fec-b041-a6f3615d7949-metrics-tls\") pod \"dns-default-nn6k7\" (UID: \"3b9c6f82-5df3-4fec-b041-a6f3615d7949\") " pod="openshift-dns/dns-default-nn6k7"
Jul 20 19:57:00.922076 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:00.922032    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 19:57:00.922076 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:00.922037    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 19:57:00.922076 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:00.922055    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b44dfd7b9-5lwkw: secret "image-registry-tls" not found
Jul 20 19:57:00.922164 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:00.922083    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b9c6f82-5df3-4fec-b041-a6f3615d7949-metrics-tls podName:3b9c6f82-5df3-4fec-b041-a6f3615d7949 nodeName:}" failed. No retries permitted until 2026-07-20 19:57:02.922070356 +0000 UTC m=+38.186140380 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b9c6f82-5df3-4fec-b041-a6f3615d7949-metrics-tls") pod "dns-default-nn6k7" (UID: "3b9c6f82-5df3-4fec-b041-a6f3615d7949") : secret "dns-default-metrics-tls" not found
Jul 20 19:57:00.922164 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:00.922096    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-tls podName:e441b5f3-c7c5-4981-bc51-c797d2b43002 nodeName:}" failed. No retries permitted until 2026-07-20 19:57:02.922090408 +0000 UTC m=+38.186160432 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-tls") pod "image-registry-5b44dfd7b9-5lwkw" (UID: "e441b5f3-c7c5-4981-bc51-c797d2b43002") : secret "image-registry-tls" not found
Jul 20 19:57:01.518133 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:01.518096    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xvdd8" event={"ID":"9ac271c1-f11b-4518-a768-697ed858df99","Type":"ContainerStarted","Data":"6b4a9fa9fae0c2883d2af7c8c7399135ed50abcbcf973a4c7b06c5d9cfa76a61"}
Jul 20 19:57:01.521272 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:01.521240    2580 generic.go:356] "Generic (PLEG): container finished" podID="bfaea5fd-9480-4cd0-b133-217c1eb6069a" containerID="f8d54411fe140a80f27c017728ee19672a89d3e7fae25fb2309fde1f0adb4bea" exitCode=0
Jul 20 19:57:01.521393 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:01.521289    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b94k5" event={"ID":"bfaea5fd-9480-4cd0-b133-217c1eb6069a","Type":"ContainerDied","Data":"f8d54411fe140a80f27c017728ee19672a89d3e7fae25fb2309fde1f0adb4bea"}
Jul 20 19:57:02.171604 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.171387    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mmxqq"]
Jul 20 19:57:02.174998 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.174975    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:02.178579 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.178553    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 20 19:57:02.179586 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.179563    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-84blb\""
Jul 20 19:57:02.180109 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.180082    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 20 19:57:02.180109 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.180099    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 20 19:57:02.180353 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.180340    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 20 19:57:02.209333 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.209171    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mmxqq"]
Jul 20 19:57:02.334346 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.334308    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/038028aa-4904-47b7-8284-0197ad123412-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:02.334501 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.334359    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/038028aa-4904-47b7-8284-0197ad123412-crio-socket\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:02.334501 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.334452    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/038028aa-4904-47b7-8284-0197ad123412-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:02.334625 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.334521    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/038028aa-4904-47b7-8284-0197ad123412-data-volume\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:02.334625 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.334556    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qxhf\" (UniqueName: \"kubernetes.io/projected/038028aa-4904-47b7-8284-0197ad123412-kube-api-access-7qxhf\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:02.435707 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.435621    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/038028aa-4904-47b7-8284-0197ad123412-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:02.435707 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.435680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/038028aa-4904-47b7-8284-0197ad123412-crio-socket\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:02.435986 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.435741    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/038028aa-4904-47b7-8284-0197ad123412-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:02.435986 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.435770    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/038028aa-4904-47b7-8284-0197ad123412-data-volume\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:02.435986 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.435791    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qxhf\" (UniqueName: \"kubernetes.io/projected/038028aa-4904-47b7-8284-0197ad123412-kube-api-access-7qxhf\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:02.435986 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:02.435790    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 19:57:02.435986 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:02.435838    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 19:57:02.435986 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:02.435897    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/038028aa-4904-47b7-8284-0197ad123412-kube-rbac-proxy-cm podName:038028aa-4904-47b7-8284-0197ad123412 nodeName:}" failed. No retries permitted until 2026-07-20 19:57:02.935881773 +0000 UTC m=+38.199951796 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/038028aa-4904-47b7-8284-0197ad123412-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mmxqq" (UID: "038028aa-4904-47b7-8284-0197ad123412") : configmap "kube-rbac-proxy" not found
Jul 20 19:57:02.435986 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:02.435918    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/038028aa-4904-47b7-8284-0197ad123412-insights-runtime-extractor-tls podName:038028aa-4904-47b7-8284-0197ad123412 nodeName:}" failed. No retries permitted until 2026-07-20 19:57:02.935907876 +0000 UTC m=+38.199977927 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/038028aa-4904-47b7-8284-0197ad123412-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mmxqq" (UID: "038028aa-4904-47b7-8284-0197ad123412") : secret "insights-runtime-extractor-tls" not found
Jul 20 19:57:02.435986 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.435931    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/038028aa-4904-47b7-8284-0197ad123412-crio-socket\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:02.436355 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.436267    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/038028aa-4904-47b7-8284-0197ad123412-data-volume\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:02.448401 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.448369    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qxhf\" (UniqueName: \"kubernetes.io/projected/038028aa-4904-47b7-8284-0197ad123412-kube-api-access-7qxhf\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:02.527192 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.527158    2580 generic.go:356] "Generic (PLEG): container finished" podID="bfaea5fd-9480-4cd0-b133-217c1eb6069a" containerID="b3cd5f2c726c37099cb945d676f6b78d697b9c7f789dbd75370ca506c9344c3b" exitCode=0
Jul 20 19:57:02.527353 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.527213    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b94k5" event={"ID":"bfaea5fd-9480-4cd0-b133-217c1eb6069a","Type":"ContainerDied","Data":"b3cd5f2c726c37099cb945d676f6b78d697b9c7f789dbd75370ca506c9344c3b"}
Jul 20 19:57:02.940475 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.940433    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/038028aa-4904-47b7-8284-0197ad123412-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:02.940475 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.940484    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-tls\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:57:02.940731 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.940538    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b9c6f82-5df3-4fec-b041-a6f3615d7949-metrics-tls\") pod \"dns-default-nn6k7\" (UID: \"3b9c6f82-5df3-4fec-b041-a6f3615d7949\") " pod="openshift-dns/dns-default-nn6k7"
Jul 20 19:57:02.940731 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:02.940568    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/038028aa-4904-47b7-8284-0197ad123412-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:02.940731 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:02.940568    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 19:57:02.940731 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:02.940652    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/038028aa-4904-47b7-8284-0197ad123412-kube-rbac-proxy-cm podName:038028aa-4904-47b7-8284-0197ad123412 nodeName:}" failed. No retries permitted until 2026-07-20 19:57:03.940621401 +0000 UTC m=+39.204691427 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/038028aa-4904-47b7-8284-0197ad123412-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mmxqq" (UID: "038028aa-4904-47b7-8284-0197ad123412") : configmap "kube-rbac-proxy" not found
Jul 20 19:57:02.940731 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:02.940663    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 19:57:02.940731 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:02.940668    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 19:57:02.940731 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:02.940687    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b44dfd7b9-5lwkw: secret "image-registry-tls" not found
Jul 20 19:57:02.940731 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:02.940716    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/038028aa-4904-47b7-8284-0197ad123412-insights-runtime-extractor-tls podName:038028aa-4904-47b7-8284-0197ad123412 nodeName:}" failed. No retries permitted until 2026-07-20 19:57:03.940699901 +0000 UTC m=+39.204769928 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/038028aa-4904-47b7-8284-0197ad123412-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mmxqq" (UID: "038028aa-4904-47b7-8284-0197ad123412") : secret "insights-runtime-extractor-tls" not found
Jul 20 19:57:02.940731 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:02.940663    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 19:57:02.940731 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:02.940731    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-tls podName:e441b5f3-c7c5-4981-bc51-c797d2b43002 nodeName:}" failed. No retries permitted until 2026-07-20 19:57:06.940724368 +0000 UTC m=+42.204794392 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-tls") pod "image-registry-5b44dfd7b9-5lwkw" (UID: "e441b5f3-c7c5-4981-bc51-c797d2b43002") : secret "image-registry-tls" not found
Jul 20 19:57:02.941134 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:02.940755    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b9c6f82-5df3-4fec-b041-a6f3615d7949-metrics-tls podName:3b9c6f82-5df3-4fec-b041-a6f3615d7949 nodeName:}" failed. No retries permitted until 2026-07-20 19:57:06.940739417 +0000 UTC m=+42.204809440 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b9c6f82-5df3-4fec-b041-a6f3615d7949-metrics-tls") pod "dns-default-nn6k7" (UID: "3b9c6f82-5df3-4fec-b041-a6f3615d7949") : secret "dns-default-metrics-tls" not found
Jul 20 19:57:03.236914 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:03.236882    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vc57q"]
Jul 20 19:57:03.253474 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:03.253450    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vc57q"]
Jul 20 19:57:03.253603 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:03.253582    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vc57q"
Jul 20 19:57:03.256721 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:03.256699    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-qccv5\""
Jul 20 19:57:03.256721 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:03.256712    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 20 19:57:03.256887 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:03.256732    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 20 19:57:03.344863 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:03.344812    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k8k7\" (UniqueName: \"kubernetes.io/projected/ab77f7a5-598d-4e32-be06-93185adc3e76-kube-api-access-5k8k7\") pod \"migrator-7cc9c7ffd7-vc57q\" (UID: \"ab77f7a5-598d-4e32-be06-93185adc3e76\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vc57q"
Jul 20 19:57:03.446083 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:03.446048    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5k8k7\" (UniqueName: \"kubernetes.io/projected/ab77f7a5-598d-4e32-be06-93185adc3e76-kube-api-access-5k8k7\") pod \"migrator-7cc9c7ffd7-vc57q\" (UID: \"ab77f7a5-598d-4e32-be06-93185adc3e76\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vc57q"
Jul 20 19:57:03.455273 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:03.455250    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k8k7\" (UniqueName: \"kubernetes.io/projected/ab77f7a5-598d-4e32-be06-93185adc3e76-kube-api-access-5k8k7\") pod \"migrator-7cc9c7ffd7-vc57q\" (UID: \"ab77f7a5-598d-4e32-be06-93185adc3e76\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vc57q"
Jul 20 19:57:03.530992 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:03.530881    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xvdd8" event={"ID":"9ac271c1-f11b-4518-a768-697ed858df99","Type":"ContainerStarted","Data":"6e22fb498777d93cf550607d3a2c95e86ec798e8f3ebc5235a447905e0810d6d"}
Jul 20 19:57:03.537103 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:03.537074    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b94k5" event={"ID":"bfaea5fd-9480-4cd0-b133-217c1eb6069a","Type":"ContainerStarted","Data":"140003292d7c9e5ed01e7b784e370d2b403f3de77de2ac0ee6c41433740e45ad"}
Jul 20 19:57:03.549725 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:03.549686    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xvdd8" podStartSLOduration=2.164709117 podStartE2EDuration="4.549674848s" totalImagesPullingTime="2.384965731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:56:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:57:03.549529828 +0000 UTC m=+38.813599874" watchObservedRunningTime="2026-07-20 19:57:03.549674848 +0000 UTC m=+38.813744893"
Jul 20 19:57:03.562440 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:03.562406    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vc57q"
Jul 20 19:57:03.572938 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:03.572901    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-b94k5" podStartSLOduration=7.9226980099999995 podStartE2EDuration="38.572887485s" totalImagesPullingTime="30.650189475s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:56:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:57:03.571147677 +0000 UTC m=+38.835217724" watchObservedRunningTime="2026-07-20 19:57:03.572887485 +0000 UTC m=+38.836957530"
Jul 20 19:57:03.757355 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:03.757321    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vc57q"]
Jul 20 19:57:03.760081 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:57:03.760059    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab77f7a5_598d_4e32_be06_93185adc3e76.slice/crio-9355efd489e92b7d4a8f74388ce60175022e3d24f55ac346422ebb92ad883778 WatchSource:0}: Error finding container 9355efd489e92b7d4a8f74388ce60175022e3d24f55ac346422ebb92ad883778: Status 404 returned error can't find the container with id 9355efd489e92b7d4a8f74388ce60175022e3d24f55ac346422ebb92ad883778
Jul 20 19:57:03.796741 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:03.796683    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nngvn_a164b5a7-e049-42a5-8f89-915704f82f68/dns-node-resolver/0.log"
Jul 20 19:57:03.951366 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:03.951327    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/038028aa-4904-47b7-8284-0197ad123412-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:03.951527 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:03.951411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/038028aa-4904-47b7-8284-0197ad123412-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:03.951527 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:03.951453    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 19:57:03.951527 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:03.951513    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 19:57:03.951527 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:03.951525    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/038028aa-4904-47b7-8284-0197ad123412-kube-rbac-proxy-cm podName:038028aa-4904-47b7-8284-0197ad123412 nodeName:}" failed. No retries permitted until 2026-07-20 19:57:05.951507173 +0000 UTC m=+41.215577211 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/038028aa-4904-47b7-8284-0197ad123412-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mmxqq" (UID: "038028aa-4904-47b7-8284-0197ad123412") : configmap "kube-rbac-proxy" not found
Jul 20 19:57:03.951654 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:03.951549    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/038028aa-4904-47b7-8284-0197ad123412-insights-runtime-extractor-tls podName:038028aa-4904-47b7-8284-0197ad123412 nodeName:}" failed. No retries permitted until 2026-07-20 19:57:05.951539314 +0000 UTC m=+41.215609342 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/038028aa-4904-47b7-8284-0197ad123412-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mmxqq" (UID: "038028aa-4904-47b7-8284-0197ad123412") : secret "insights-runtime-extractor-tls" not found
Jul 20 19:57:04.391990 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:04.391942    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-vww5k"]
Jul 20 19:57:04.405742 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:04.405713    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-vww5k"
Jul 20 19:57:04.406301 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:04.406138    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-vww5k"]
Jul 20 19:57:04.410413 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:04.410386    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 20 19:57:04.410542 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:04.410430    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 20 19:57:04.411596 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:04.411572    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 20 19:57:04.411596 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:04.411594    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 20 19:57:04.411750 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:04.411641    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-xt6pv\""
Jul 20 19:57:04.540989 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:04.540946    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vc57q" event={"ID":"ab77f7a5-598d-4e32-be06-93185adc3e76","Type":"ContainerStarted","Data":"9355efd489e92b7d4a8f74388ce60175022e3d24f55ac346422ebb92ad883778"}
Jul 20 19:57:04.556856 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:04.556825    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c1e8bc55-88d9-4193-90b1-c306aa2c91c4-signing-key\") pod \"service-ca-58598695f7-vww5k\" (UID: \"c1e8bc55-88d9-4193-90b1-c306aa2c91c4\") " pod="openshift-service-ca/service-ca-58598695f7-vww5k"
Jul 20 19:57:04.556856 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:04.556859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2bqz\" (UniqueName: \"kubernetes.io/projected/c1e8bc55-88d9-4193-90b1-c306aa2c91c4-kube-api-access-g2bqz\") pod \"service-ca-58598695f7-vww5k\" (UID: \"c1e8bc55-88d9-4193-90b1-c306aa2c91c4\") " pod="openshift-service-ca/service-ca-58598695f7-vww5k"
Jul 20 19:57:04.557105 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:04.556881    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c1e8bc55-88d9-4193-90b1-c306aa2c91c4-signing-cabundle\") pod \"service-ca-58598695f7-vww5k\" (UID: \"c1e8bc55-88d9-4193-90b1-c306aa2c91c4\") " pod="openshift-service-ca/service-ca-58598695f7-vww5k"
Jul 20 19:57:04.658009 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:04.657899    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c1e8bc55-88d9-4193-90b1-c306aa2c91c4-signing-cabundle\") pod \"service-ca-58598695f7-vww5k\" (UID: \"c1e8bc55-88d9-4193-90b1-c306aa2c91c4\") " pod="openshift-service-ca/service-ca-58598695f7-vww5k"
Jul 20 19:57:04.658279 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:04.658239    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c1e8bc55-88d9-4193-90b1-c306aa2c91c4-signing-key\") pod \"service-ca-58598695f7-vww5k\" (UID: \"c1e8bc55-88d9-4193-90b1-c306aa2c91c4\") " pod="openshift-service-ca/service-ca-58598695f7-vww5k"
Jul 20 19:57:04.658279 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:04.658282    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2bqz\" (UniqueName: \"kubernetes.io/projected/c1e8bc55-88d9-4193-90b1-c306aa2c91c4-kube-api-access-g2bqz\") pod \"service-ca-58598695f7-vww5k\" (UID: \"c1e8bc55-88d9-4193-90b1-c306aa2c91c4\") " pod="openshift-service-ca/service-ca-58598695f7-vww5k"
Jul 20 19:57:04.658873 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:04.658846    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c1e8bc55-88d9-4193-90b1-c306aa2c91c4-signing-cabundle\") pod \"service-ca-58598695f7-vww5k\" (UID: \"c1e8bc55-88d9-4193-90b1-c306aa2c91c4\") " pod="openshift-service-ca/service-ca-58598695f7-vww5k"
Jul 20 19:57:04.660761 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:04.660740    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c1e8bc55-88d9-4193-90b1-c306aa2c91c4-signing-key\") pod \"service-ca-58598695f7-vww5k\" (UID: \"c1e8bc55-88d9-4193-90b1-c306aa2c91c4\") " pod="openshift-service-ca/service-ca-58598695f7-vww5k"
Jul 20 19:57:04.667461 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:04.667440    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2bqz\" (UniqueName: \"kubernetes.io/projected/c1e8bc55-88d9-4193-90b1-c306aa2c91c4-kube-api-access-g2bqz\") pod \"service-ca-58598695f7-vww5k\" (UID: \"c1e8bc55-88d9-4193-90b1-c306aa2c91c4\") " pod="openshift-service-ca/service-ca-58598695f7-vww5k"
Jul 20 19:57:04.717317 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:04.717284    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-vww5k"
Jul 20 19:57:04.865135 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:04.865094    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-vww5k"]
Jul 20 19:57:04.995385 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:04.995311    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w5555_b4c1d9fe-73ea-4091-9855-b9f6970d176e/node-ca/0.log"
Jul 20 19:57:05.114464 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:57:05.114427    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1e8bc55_88d9_4193_90b1_c306aa2c91c4.slice/crio-dfc7085ef4772a75315a8e53638c7bbc977b6083591beb1333443cb6bd73690f WatchSource:0}: Error finding container dfc7085ef4772a75315a8e53638c7bbc977b6083591beb1333443cb6bd73690f: Status 404 returned error can't find the container with id dfc7085ef4772a75315a8e53638c7bbc977b6083591beb1333443cb6bd73690f
Jul 20 19:57:05.544132 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:05.544104    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-vww5k" event={"ID":"c1e8bc55-88d9-4193-90b1-c306aa2c91c4","Type":"ContainerStarted","Data":"dfc7085ef4772a75315a8e53638c7bbc977b6083591beb1333443cb6bd73690f"}
Jul 20 19:57:05.597668 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:05.597643    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xvdd8_9ac271c1-f11b-4518-a768-697ed858df99/serve-healthcheck-canary/0.log"
Jul 20 19:57:05.971029 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:05.970986    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/038028aa-4904-47b7-8284-0197ad123412-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:05.971200 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:05.971036    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 19:57:05.971200 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:05.971100    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/038028aa-4904-47b7-8284-0197ad123412-kube-rbac-proxy-cm podName:038028aa-4904-47b7-8284-0197ad123412 nodeName:}" failed. No retries permitted until 2026-07-20 19:57:09.971083075 +0000 UTC m=+45.235153121 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/038028aa-4904-47b7-8284-0197ad123412-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mmxqq" (UID: "038028aa-4904-47b7-8284-0197ad123412") : configmap "kube-rbac-proxy" not found
Jul 20 19:57:05.971200 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:05.971132    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/038028aa-4904-47b7-8284-0197ad123412-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:05.971305 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:05.971223    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 19:57:05.971305 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:05.971257    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/038028aa-4904-47b7-8284-0197ad123412-insights-runtime-extractor-tls podName:038028aa-4904-47b7-8284-0197ad123412 nodeName:}" failed. No retries permitted until 2026-07-20 19:57:09.971249742 +0000 UTC m=+45.235319766 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/038028aa-4904-47b7-8284-0197ad123412-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mmxqq" (UID: "038028aa-4904-47b7-8284-0197ad123412") : secret "insights-runtime-extractor-tls" not found
Jul 20 19:57:06.548665 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:06.548632    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vc57q" event={"ID":"ab77f7a5-598d-4e32-be06-93185adc3e76","Type":"ContainerStarted","Data":"7a0ae23d15e1bca103b11f2742ee5cd10fa968b7757876d482534650ffa616e6"}
Jul 20 19:57:06.549154 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:06.548687    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vc57q" event={"ID":"ab77f7a5-598d-4e32-be06-93185adc3e76","Type":"ContainerStarted","Data":"5b48d935b7a97ee5b8760e515e7e915c14f26c62667674c46624a18bbfe9c111"}
Jul 20 19:57:06.568519 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:06.568469    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vc57q" podStartSLOduration=1.851287939 podStartE2EDuration="3.568455429s" totalImagesPullingTime="1.71716749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:57:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:57:06.568170151 +0000 UTC m=+41.832240198" watchObservedRunningTime="2026-07-20 19:57:06.568455429 +0000 UTC m=+41.832525475"
Jul 20 19:57:06.980209 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:06.980114    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-tls\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:57:06.980209 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:06.980184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b9c6f82-5df3-4fec-b041-a6f3615d7949-metrics-tls\") pod \"dns-default-nn6k7\" (UID: \"3b9c6f82-5df3-4fec-b041-a6f3615d7949\") " pod="openshift-dns/dns-default-nn6k7"
Jul 20 19:57:06.980432 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:06.980284    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 19:57:06.980432 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:06.980292    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 19:57:06.980432 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:06.980301    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b44dfd7b9-5lwkw: secret "image-registry-tls" not found
Jul 20 19:57:06.980432 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:06.980352    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-tls podName:e441b5f3-c7c5-4981-bc51-c797d2b43002 nodeName:}" failed. No retries permitted until 2026-07-20 19:57:14.980336042 +0000 UTC m=+50.244406081 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-tls") pod "image-registry-5b44dfd7b9-5lwkw" (UID: "e441b5f3-c7c5-4981-bc51-c797d2b43002") : secret "image-registry-tls" not found
Jul 20 19:57:06.980432 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:06.980372    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b9c6f82-5df3-4fec-b041-a6f3615d7949-metrics-tls podName:3b9c6f82-5df3-4fec-b041-a6f3615d7949 nodeName:}" failed. No retries permitted until 2026-07-20 19:57:14.980363453 +0000 UTC m=+50.244433478 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b9c6f82-5df3-4fec-b041-a6f3615d7949-metrics-tls") pod "dns-default-nn6k7" (UID: "3b9c6f82-5df3-4fec-b041-a6f3615d7949") : secret "dns-default-metrics-tls" not found
Jul 20 19:57:08.554674 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:08.554632    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-vww5k" event={"ID":"c1e8bc55-88d9-4193-90b1-c306aa2c91c4","Type":"ContainerStarted","Data":"a53cd60fa9a59e3146a5b1d2af652eec1d6e559c132c1ae7a5b39ed8834fe2c9"}
Jul 20 19:57:10.006343 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:10.006307    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/038028aa-4904-47b7-8284-0197ad123412-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:10.006792 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:10.006389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/038028aa-4904-47b7-8284-0197ad123412-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:10.006792 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:10.006477    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 19:57:10.006792 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:10.006528    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/038028aa-4904-47b7-8284-0197ad123412-insights-runtime-extractor-tls podName:038028aa-4904-47b7-8284-0197ad123412 nodeName:}" failed. No retries permitted until 2026-07-20 19:57:18.006515452 +0000 UTC m=+53.270585476 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/038028aa-4904-47b7-8284-0197ad123412-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mmxqq" (UID: "038028aa-4904-47b7-8284-0197ad123412") : secret "insights-runtime-extractor-tls" not found
Jul 20 19:57:10.007009 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:10.006987    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/038028aa-4904-47b7-8284-0197ad123412-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:15.049674 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:15.049630    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-tls\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:57:15.050071 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:15.049693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b9c6f82-5df3-4fec-b041-a6f3615d7949-metrics-tls\") pod \"dns-default-nn6k7\" (UID: \"3b9c6f82-5df3-4fec-b041-a6f3615d7949\") " pod="openshift-dns/dns-default-nn6k7"
Jul 20 19:57:15.051999 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:15.051972    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-tls\") pod \"image-registry-5b44dfd7b9-5lwkw\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") " pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:57:15.051999 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:15.051981    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b9c6f82-5df3-4fec-b041-a6f3615d7949-metrics-tls\") pod \"dns-default-nn6k7\" (UID: \"3b9c6f82-5df3-4fec-b041-a6f3615d7949\") " pod="openshift-dns/dns-default-nn6k7"
Jul 20 19:57:15.057969 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:15.057939    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:57:15.072082 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:15.072048    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nn6k7"
Jul 20 19:57:15.240020 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:15.239949    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-vww5k" podStartSLOduration=8.586494783 podStartE2EDuration="11.23993033s" totalImagesPullingTime="2.653435547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:57:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:57:08.57185086 +0000 UTC m=+43.835920905" watchObservedRunningTime="2026-07-20 19:57:15.23993033 +0000 UTC m=+50.504000375"
Jul 20 19:57:15.240286 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:15.240264    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"]
Jul 20 19:57:15.243338 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:57:15.243311    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode441b5f3_c7c5_4981_bc51_c797d2b43002.slice/crio-62d7c525a04ecbdeb247d48285c253819b9c101327d46bd890fffa537993273f WatchSource:0}: Error finding container 62d7c525a04ecbdeb247d48285c253819b9c101327d46bd890fffa537993273f: Status 404 returned error can't find the container with id 62d7c525a04ecbdeb247d48285c253819b9c101327d46bd890fffa537993273f
Jul 20 19:57:15.257639 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:15.257616    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nn6k7"]
Jul 20 19:57:15.260194 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:57:15.260174    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b9c6f82_5df3_4fec_b041_a6f3615d7949.slice/crio-97c2cb83bbf07fef0d683ae9f61fb3e38c44fc6cde162a36f59e5d3f3b8321eb WatchSource:0}: Error finding container 97c2cb83bbf07fef0d683ae9f61fb3e38c44fc6cde162a36f59e5d3f3b8321eb: Status 404 returned error can't find the container with id 97c2cb83bbf07fef0d683ae9f61fb3e38c44fc6cde162a36f59e5d3f3b8321eb
Jul 20 19:57:15.575499 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:15.575402    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nn6k7" event={"ID":"3b9c6f82-5df3-4fec-b041-a6f3615d7949","Type":"ContainerStarted","Data":"97c2cb83bbf07fef0d683ae9f61fb3e38c44fc6cde162a36f59e5d3f3b8321eb"}
Jul 20 19:57:15.576729 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:15.576706    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw" event={"ID":"e441b5f3-c7c5-4981-bc51-c797d2b43002","Type":"ContainerStarted","Data":"5ec4714ce601e3880d7f727f6b2f4ce9b7c84466a3ff7106c87a5ebf63deef5b"}
Jul 20 19:57:15.576850 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:15.576748    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw" event={"ID":"e441b5f3-c7c5-4981-bc51-c797d2b43002","Type":"ContainerStarted","Data":"62d7c525a04ecbdeb247d48285c253819b9c101327d46bd890fffa537993273f"}
Jul 20 19:57:15.576850 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:15.576844    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:57:15.609652 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:15.609604    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw" podStartSLOduration=34.609586137 podStartE2EDuration="34.609586137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:56:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:57:15.608979408 +0000 UTC m=+50.873049452" watchObservedRunningTime="2026-07-20 19:57:15.609586137 +0000 UTC m=+50.873656204"
Jul 20 19:57:18.073162 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:18.073124    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/038028aa-4904-47b7-8284-0197ad123412-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:18.075406 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:18.075380    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/038028aa-4904-47b7-8284-0197ad123412-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmxqq\" (UID: \"038028aa-4904-47b7-8284-0197ad123412\") " pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:18.086361 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:18.086340    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mmxqq"
Jul 20 19:57:18.230328 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:18.230304    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mmxqq"]
Jul 20 19:57:18.232740 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:57:18.232718    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod038028aa_4904_47b7_8284_0197ad123412.slice/crio-2be95c278955c4f2226f1ecc0fe8448e7a1f36b8cef69cf01523f1fb607d1909 WatchSource:0}: Error finding container 2be95c278955c4f2226f1ecc0fe8448e7a1f36b8cef69cf01523f1fb607d1909: Status 404 returned error can't find the container with id 2be95c278955c4f2226f1ecc0fe8448e7a1f36b8cef69cf01523f1fb607d1909
Jul 20 19:57:18.587905 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:18.587869    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mmxqq" event={"ID":"038028aa-4904-47b7-8284-0197ad123412","Type":"ContainerStarted","Data":"2be95c278955c4f2226f1ecc0fe8448e7a1f36b8cef69cf01523f1fb607d1909"}
Jul 20 19:57:18.589494 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:18.589465    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nn6k7" event={"ID":"3b9c6f82-5df3-4fec-b041-a6f3615d7949","Type":"ContainerStarted","Data":"f680dd5d209277fce53bede3e0bd44ed84db7798c8a8f0909132302ff07700fd"}
Jul 20 19:57:18.589626 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:18.589508    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nn6k7" event={"ID":"3b9c6f82-5df3-4fec-b041-a6f3615d7949","Type":"ContainerStarted","Data":"0dd3dc4ec527463a505fd553fa06585f4816a056e2e2996b60107324ea31ed6e"}
Jul 20 19:57:18.589626 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:18.589613    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nn6k7"
Jul 20 19:57:18.611505 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:18.611454    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nn6k7" podStartSLOduration=17.224039331 podStartE2EDuration="19.61143836s" totalImagesPullingTime="2.387399029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:56:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:57:18.609606141 +0000 UTC m=+53.873676187" watchObservedRunningTime="2026-07-20 19:57:18.61143836 +0000 UTC m=+53.875508451"
Jul 20 19:57:23.611873 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:23.611842    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mmxqq" event={"ID":"038028aa-4904-47b7-8284-0197ad123412","Type":"ContainerStarted","Data":"33e253c4508a7a886af4a241de8f97ddd1442249130215b8ca32a1f4b4b09318"}
Jul 20 19:57:25.520880 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.520843    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8ldg4"
Jul 20 19:57:25.534120 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.534085    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f58cbd79b-dk6vc"]
Jul 20 19:57:25.616113 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.616048    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"]
Jul 20 19:57:25.616293 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.616138    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f58cbd79b-dk6vc"
Jul 20 19:57:25.619162 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.619129    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 20 19:57:25.619653 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.619513    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 20 19:57:25.619653 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.619521    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-kv46x\""
Jul 20 19:57:25.619653 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.619554    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 20 19:57:25.619653 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.619579    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 20 19:57:25.635474 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.635445    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f58cbd79b-dk6vc"]
Jul 20 19:57:25.635474 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.635484    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"]
Jul 20 19:57:25.635673 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.635514    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"]
Jul 20 19:57:25.635673 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.635588    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"
Jul 20 19:57:25.636702 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.636676    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-77c574ccb-8757d"]
Jul 20 19:57:25.638428 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.638397    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 20 19:57:25.638758 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.638735    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 20 19:57:25.639278 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.639091    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 20 19:57:25.639278 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.639174    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 20 19:57:25.651819 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.651757    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.673943 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.673922    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77c574ccb-8757d"]
Jul 20 19:57:25.741022 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.740989    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp79m\" (UniqueName: \"kubernetes.io/projected/5c38b80c-f78b-43fc-b78c-bb7c9e8c716f-kube-api-access-xp79m\") pod \"cluster-proxy-proxy-agent-5b64bfddb7-6tg4v\" (UID: \"5c38b80c-f78b-43fc-b78c-bb7c9e8c716f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"
Jul 20 19:57:25.741203 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.741144    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5c38b80c-f78b-43fc-b78c-bb7c9e8c716f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5b64bfddb7-6tg4v\" (UID: \"5c38b80c-f78b-43fc-b78c-bb7c9e8c716f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"
Jul 20 19:57:25.741203 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.741182    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5c38b80c-f78b-43fc-b78c-bb7c9e8c716f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5b64bfddb7-6tg4v\" (UID: \"5c38b80c-f78b-43fc-b78c-bb7c9e8c716f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"
Jul 20 19:57:25.741203 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.741202    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-bound-sa-token\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.741382 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.741222    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-registry-certificates\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.741382 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.741250    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmmdq\" (UniqueName: \"kubernetes.io/projected/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-kube-api-access-gmmdq\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.741382 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.741298    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-registry-tls\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.741382 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.741321    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/01a57517-9938-4b53-9982-4f6d7986abc0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f58cbd79b-dk6vc\" (UID: \"01a57517-9938-4b53-9982-4f6d7986abc0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f58cbd79b-dk6vc"
Jul 20 19:57:25.741382 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.741339    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c38b80c-f78b-43fc-b78c-bb7c9e8c716f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5b64bfddb7-6tg4v\" (UID: \"5c38b80c-f78b-43fc-b78c-bb7c9e8c716f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"
Jul 20 19:57:25.741382 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.741354    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-image-registry-private-configuration\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.741607 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.741430    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5c38b80c-f78b-43fc-b78c-bb7c9e8c716f-hub\") pod \"cluster-proxy-proxy-agent-5b64bfddb7-6tg4v\" (UID: \"5c38b80c-f78b-43fc-b78c-bb7c9e8c716f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"
Jul 20 19:57:25.741607 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.741502    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-trusted-ca\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.741607 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.741591    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-ca-trust-extracted\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.741735 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.741642    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q766m\" (UniqueName: \"kubernetes.io/projected/01a57517-9938-4b53-9982-4f6d7986abc0-kube-api-access-q766m\") pod \"managed-serviceaccount-addon-agent-6f58cbd79b-dk6vc\" (UID: \"01a57517-9938-4b53-9982-4f6d7986abc0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f58cbd79b-dk6vc"
Jul 20 19:57:25.741735 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.741670    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5c38b80c-f78b-43fc-b78c-bb7c9e8c716f-ca\") pod \"cluster-proxy-proxy-agent-5b64bfddb7-6tg4v\" (UID: \"5c38b80c-f78b-43fc-b78c-bb7c9e8c716f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"
Jul 20 19:57:25.741735 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.741704    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-installation-pull-secrets\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.842241 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.842212    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp79m\" (UniqueName: \"kubernetes.io/projected/5c38b80c-f78b-43fc-b78c-bb7c9e8c716f-kube-api-access-xp79m\") pod \"cluster-proxy-proxy-agent-5b64bfddb7-6tg4v\" (UID: \"5c38b80c-f78b-43fc-b78c-bb7c9e8c716f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"
Jul 20 19:57:25.842432 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.842259    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5c38b80c-f78b-43fc-b78c-bb7c9e8c716f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5b64bfddb7-6tg4v\" (UID: \"5c38b80c-f78b-43fc-b78c-bb7c9e8c716f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"
Jul 20 19:57:25.842432 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.842284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5c38b80c-f78b-43fc-b78c-bb7c9e8c716f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5b64bfddb7-6tg4v\" (UID: \"5c38b80c-f78b-43fc-b78c-bb7c9e8c716f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"
Jul 20 19:57:25.842432 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.842307    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-bound-sa-token\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.842432 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.842332    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-registry-certificates\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.842432 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.842359    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmmdq\" (UniqueName: \"kubernetes.io/projected/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-kube-api-access-gmmdq\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.842432 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.842401    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-registry-tls\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.842432 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.842429    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/01a57517-9938-4b53-9982-4f6d7986abc0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f58cbd79b-dk6vc\" (UID: \"01a57517-9938-4b53-9982-4f6d7986abc0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f58cbd79b-dk6vc"
Jul 20 19:57:25.843035 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.842456    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c38b80c-f78b-43fc-b78c-bb7c9e8c716f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5b64bfddb7-6tg4v\" (UID: \"5c38b80c-f78b-43fc-b78c-bb7c9e8c716f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"
Jul 20 19:57:25.843035 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.842482    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-image-registry-private-configuration\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.843035 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.842511    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5c38b80c-f78b-43fc-b78c-bb7c9e8c716f-hub\") pod \"cluster-proxy-proxy-agent-5b64bfddb7-6tg4v\" (UID: \"5c38b80c-f78b-43fc-b78c-bb7c9e8c716f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"
Jul 20 19:57:25.843035 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.842540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-trusted-ca\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.843035 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.842578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-ca-trust-extracted\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.843035 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.842617    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q766m\" (UniqueName: \"kubernetes.io/projected/01a57517-9938-4b53-9982-4f6d7986abc0-kube-api-access-q766m\") pod \"managed-serviceaccount-addon-agent-6f58cbd79b-dk6vc\" (UID: \"01a57517-9938-4b53-9982-4f6d7986abc0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f58cbd79b-dk6vc"
Jul 20 19:57:25.843035 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.842641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5c38b80c-f78b-43fc-b78c-bb7c9e8c716f-ca\") pod \"cluster-proxy-proxy-agent-5b64bfddb7-6tg4v\" (UID: \"5c38b80c-f78b-43fc-b78c-bb7c9e8c716f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"
Jul 20 19:57:25.843035 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.842675    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-installation-pull-secrets\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.844332 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.843293    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-registry-certificates\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.844332 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.843526    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c38b80c-f78b-43fc-b78c-bb7c9e8c716f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5b64bfddb7-6tg4v\" (UID: \"5c38b80c-f78b-43fc-b78c-bb7c9e8c716f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"
Jul 20 19:57:25.844431 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.844363    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-trusted-ca\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.844986 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.844897    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-ca-trust-extracted\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.846648 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.846621    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5c38b80c-f78b-43fc-b78c-bb7c9e8c716f-ca\") pod \"cluster-proxy-proxy-agent-5b64bfddb7-6tg4v\" (UID: \"5c38b80c-f78b-43fc-b78c-bb7c9e8c716f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"
Jul 20 19:57:25.846808 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.846782    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/01a57517-9938-4b53-9982-4f6d7986abc0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f58cbd79b-dk6vc\" (UID: \"01a57517-9938-4b53-9982-4f6d7986abc0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f58cbd79b-dk6vc"
Jul 20 19:57:25.847250 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.847229    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5c38b80c-f78b-43fc-b78c-bb7c9e8c716f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5b64bfddb7-6tg4v\" (UID: \"5c38b80c-f78b-43fc-b78c-bb7c9e8c716f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"
Jul 20 19:57:25.847336 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.847247    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5c38b80c-f78b-43fc-b78c-bb7c9e8c716f-hub\") pod \"cluster-proxy-proxy-agent-5b64bfddb7-6tg4v\" (UID: \"5c38b80c-f78b-43fc-b78c-bb7c9e8c716f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"
Jul 20 19:57:25.847456 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.847431    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5c38b80c-f78b-43fc-b78c-bb7c9e8c716f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5b64bfddb7-6tg4v\" (UID: \"5c38b80c-f78b-43fc-b78c-bb7c9e8c716f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"
Jul 20 19:57:25.847645 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.847621    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-image-registry-private-configuration\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.848083 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.848058    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-installation-pull-secrets\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.848912 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.848890    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-registry-tls\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.855122 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.855083    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-bound-sa-token\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.855303 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.855273    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmmdq\" (UniqueName: \"kubernetes.io/projected/903a01f6-5dc1-4d21-a0a1-0a2fac389d1c-kube-api-access-gmmdq\") pod \"image-registry-77c574ccb-8757d\" (UID: \"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c\") " pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:25.855367 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.855352    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q766m\" (UniqueName: \"kubernetes.io/projected/01a57517-9938-4b53-9982-4f6d7986abc0-kube-api-access-q766m\") pod \"managed-serviceaccount-addon-agent-6f58cbd79b-dk6vc\" (UID: \"01a57517-9938-4b53-9982-4f6d7986abc0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f58cbd79b-dk6vc"
Jul 20 19:57:25.856253 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.856229    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp79m\" (UniqueName: \"kubernetes.io/projected/5c38b80c-f78b-43fc-b78c-bb7c9e8c716f-kube-api-access-xp79m\") pod \"cluster-proxy-proxy-agent-5b64bfddb7-6tg4v\" (UID: \"5c38b80c-f78b-43fc-b78c-bb7c9e8c716f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"
Jul 20 19:57:25.933150 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.933119    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f58cbd79b-dk6vc"
Jul 20 19:57:25.946572 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.946540    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"
Jul 20 19:57:25.971789 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:25.971755    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:26.947858 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:26.947785    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v"]
Jul 20 19:57:27.054607 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:27.054574    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f58cbd79b-dk6vc"]
Jul 20 19:57:27.057276 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:57:27.057250    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01a57517_9938_4b53_9982_4f6d7986abc0.slice/crio-f9e7945a4bb808fb0d4dda0930dd5a2dd92d96292924002d65c8f8278e5d45b0 WatchSource:0}: Error finding container f9e7945a4bb808fb0d4dda0930dd5a2dd92d96292924002d65c8f8278e5d45b0: Status 404 returned error can't find the container with id f9e7945a4bb808fb0d4dda0930dd5a2dd92d96292924002d65c8f8278e5d45b0
Jul 20 19:57:27.069604 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:27.069584    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77c574ccb-8757d"]
Jul 20 19:57:27.071777 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:57:27.071754    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod903a01f6_5dc1_4d21_a0a1_0a2fac389d1c.slice/crio-e42dda2b1a3390bfaf03fc7183efa0c9d65760fa95400d18552bb6aebe4c61b5 WatchSource:0}: Error finding container e42dda2b1a3390bfaf03fc7183efa0c9d65760fa95400d18552bb6aebe4c61b5: Status 404 returned error can't find the container with id e42dda2b1a3390bfaf03fc7183efa0c9d65760fa95400d18552bb6aebe4c61b5
Jul 20 19:57:27.628732 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:27.628695    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mmxqq" event={"ID":"038028aa-4904-47b7-8284-0197ad123412","Type":"ContainerStarted","Data":"acfc4fe108406723bc33f7f0ee61939fa1cc0905295bb68038a3551dfb7d8cae"}
Jul 20 19:57:27.630125 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:27.630094    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v" event={"ID":"5c38b80c-f78b-43fc-b78c-bb7c9e8c716f","Type":"ContainerStarted","Data":"215621f3aa6c782bd2fae1d0848ad435da8ff861a33d552b5376956add876463"}
Jul 20 19:57:27.631625 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:27.631597    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77c574ccb-8757d" event={"ID":"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c","Type":"ContainerStarted","Data":"c17486090f1626818c54c17253fba5fe4be2c9c888a0df503f9fd0570543f9db"}
Jul 20 19:57:27.631734 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:27.631643    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77c574ccb-8757d" event={"ID":"903a01f6-5dc1-4d21-a0a1-0a2fac389d1c","Type":"ContainerStarted","Data":"e42dda2b1a3390bfaf03fc7183efa0c9d65760fa95400d18552bb6aebe4c61b5"}
Jul 20 19:57:27.631734 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:27.631677    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:27.632861 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:27.632831    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f58cbd79b-dk6vc" event={"ID":"01a57517-9938-4b53-9982-4f6d7986abc0","Type":"ContainerStarted","Data":"f9e7945a4bb808fb0d4dda0930dd5a2dd92d96292924002d65c8f8278e5d45b0"}
Jul 20 19:57:27.654640 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:27.654601    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-77c574ccb-8757d" podStartSLOduration=2.654585825 podStartE2EDuration="2.654585825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:57:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:57:27.653331715 +0000 UTC m=+62.917401761" watchObservedRunningTime="2026-07-20 19:57:27.654585825 +0000 UTC m=+62.918655870"
Jul 20 19:57:28.595783 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:28.595713    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nn6k7"
Jul 20 19:57:29.880793 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:29.880755    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs\") pod \"network-metrics-daemon-5778m\" (UID: \"c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1\") " pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:57:29.881295 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:29.880834    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret\") pod \"global-pull-secret-syncer-xlc4n\" (UID: \"2fbbdeaa-171d-4032-9bd3-7bbd4c009388\") " pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:57:29.883982 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:29.883941    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 20 19:57:29.884129 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:29.884002    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 20 19:57:29.894157 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:29.894134    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1-metrics-certs\") pod \"network-metrics-daemon-5778m\" (UID: \"c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1\") " pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:57:29.894277 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:29.894172    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2fbbdeaa-171d-4032-9bd3-7bbd4c009388-original-pull-secret\") pod \"global-pull-secret-syncer-xlc4n\" (UID: \"2fbbdeaa-171d-4032-9bd3-7bbd4c009388\") " pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:57:30.082487 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:30.082447    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdmfr\" (UniqueName: \"kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr\") pod \"network-check-target-5gtt2\" (UID: \"62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d\") " pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:57:30.085341 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:30.085315    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 20 19:57:30.096065 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:30.096040    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 20 19:57:30.100447 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:30.100427    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mmqwm\""
Jul 20 19:57:30.104125 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:30.104103    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xlc4n"
Jul 20 19:57:30.105857 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:30.105834    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdmfr\" (UniqueName: \"kubernetes.io/projected/62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d-kube-api-access-tdmfr\") pod \"network-check-target-5gtt2\" (UID: \"62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d\") " pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:57:30.108701 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:30.108680    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5778m"
Jul 20 19:57:30.394346 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:30.394316    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qx848\""
Jul 20 19:57:30.401997 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:30.401975    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:57:33.633626 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:33.633209    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5778m"]
Jul 20 19:57:33.658912 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:33.658883    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5778m" event={"ID":"c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1","Type":"ContainerStarted","Data":"0cdf592fabbf97cf6ca47d0c77f9891ea15b9b23a76658a797b02059aa640bdd"}
Jul 20 19:57:33.866301 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:33.866278    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xlc4n"]
Jul 20 19:57:33.869043 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:57:33.869012    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fbbdeaa_171d_4032_9bd3_7bbd4c009388.slice/crio-4f7f9d1e4d4a74f30c383ae1c5a846f1c363b2d89f04a16896972a28dcec8447 WatchSource:0}: Error finding container 4f7f9d1e4d4a74f30c383ae1c5a846f1c363b2d89f04a16896972a28dcec8447: Status 404 returned error can't find the container with id 4f7f9d1e4d4a74f30c383ae1c5a846f1c363b2d89f04a16896972a28dcec8447
Jul 20 19:57:33.872528 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:33.872508    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5gtt2"]
Jul 20 19:57:33.876480 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:57:33.876457    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62a41ec2_cdb4_4dc3_9949_eaa9456a2b9d.slice/crio-94ed8d78ab15176c502272321068b1384661168acb245e45663a018d64d90d42 WatchSource:0}: Error finding container 94ed8d78ab15176c502272321068b1384661168acb245e45663a018d64d90d42: Status 404 returned error can't find the container with id 94ed8d78ab15176c502272321068b1384661168acb245e45663a018d64d90d42
Jul 20 19:57:34.666865 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:34.666769    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xlc4n" event={"ID":"2fbbdeaa-171d-4032-9bd3-7bbd4c009388","Type":"ContainerStarted","Data":"4f7f9d1e4d4a74f30c383ae1c5a846f1c363b2d89f04a16896972a28dcec8447"}
Jul 20 19:57:34.670221 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:34.669722    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mmxqq" event={"ID":"038028aa-4904-47b7-8284-0197ad123412","Type":"ContainerStarted","Data":"8b99717ddc8c37814d0036e4989d67d992619a8adbb7708eeb5d481aff04f7b1"}
Jul 20 19:57:34.671911 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:34.671884    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5gtt2" event={"ID":"62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d","Type":"ContainerStarted","Data":"94ed8d78ab15176c502272321068b1384661168acb245e45663a018d64d90d42"}
Jul 20 19:57:34.675146 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:34.675120    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v" event={"ID":"5c38b80c-f78b-43fc-b78c-bb7c9e8c716f","Type":"ContainerStarted","Data":"6ad8d2b58f7229522e86bbd723009de3a89543849308a8c8c946a94cb682c1fe"}
Jul 20 19:57:34.675269 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:34.675158    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v" event={"ID":"5c38b80c-f78b-43fc-b78c-bb7c9e8c716f","Type":"ContainerStarted","Data":"ad1972451afcb94e769988ba039d07a3ded1304512e2d30f22a47d65f2a875c2"}
Jul 20 19:57:34.675269 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:34.675178    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v" event={"ID":"5c38b80c-f78b-43fc-b78c-bb7c9e8c716f","Type":"ContainerStarted","Data":"7d31f363be8d39f11051a85a875b6506189fd28e9e82f8d7191ef48f4910c6e8"}
Jul 20 19:57:34.678713 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:34.678672    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f58cbd79b-dk6vc" event={"ID":"01a57517-9938-4b53-9982-4f6d7986abc0","Type":"ContainerStarted","Data":"bf6e2534b359cda14997fa1e01f5cc3a3c31514d721f6fad930b2b00d198d517"}
Jul 20 19:57:34.692257 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:34.691401    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mmxqq" podStartSLOduration=17.580213207 podStartE2EDuration="32.691381444s" totalImagesPullingTime="15.111168237s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:57:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:57:34.6907216 +0000 UTC m=+69.954791649" watchObservedRunningTime="2026-07-20 19:57:34.691381444 +0000 UTC m=+69.955451492"
Jul 20 19:57:34.712824 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:34.712762    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b64bfddb7-6tg4v" podStartSLOduration=3.207735935 podStartE2EDuration="9.712739616s" totalImagesPullingTime="6.505003681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:57:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:57:34.711941457 +0000 UTC m=+69.976011514" watchObservedRunningTime="2026-07-20 19:57:34.712739616 +0000 UTC m=+69.976809664"
Jul 20 19:57:34.730725 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:34.730198    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f58cbd79b-dk6vc" podStartSLOduration=3.32525266 podStartE2EDuration="9.730175541s" totalImagesPullingTime="6.404922881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:57:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:57:34.729442051 +0000 UTC m=+69.993512097" watchObservedRunningTime="2026-07-20 19:57:34.730175541 +0000 UTC m=+69.994245590"
Jul 20 19:57:35.642519 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:35.642424    2580 patch_prober.go:28] interesting pod/image-registry-5b44dfd7b9-5lwkw container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 20 19:57:35.642519 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:35.642488    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw" podUID="e441b5f3-c7c5-4981-bc51-c797d2b43002" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 20 19:57:35.685574 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:35.685525    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5778m" event={"ID":"c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1","Type":"ContainerStarted","Data":"a540bc6887a51fd98e156e866e93cfecbb948f70b7e4f9bce62d0aea4b482fd2"}
Jul 20 19:57:35.686074 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:35.685629    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5778m" event={"ID":"c5d1940f-13c6-411c-b4bd-cbc85d6fd3f1","Type":"ContainerStarted","Data":"a86dfd8772ad45bfb64680aa5c8f6a2faec29ec1340a23f7b64cebfc0a9eb76c"}
Jul 20 19:57:35.704475 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:35.703999    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5778m" podStartSLOduration=69.215573997 podStartE2EDuration="1m10.703978571s" totalImagesPullingTime="1.488404574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:57:35.702862467 +0000 UTC m=+70.966932514" watchObservedRunningTime="2026-07-20 19:57:35.703978571 +0000 UTC m=+70.968048608"
Jul 20 19:57:39.699993 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:39.699946    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5gtt2" event={"ID":"62a41ec2-cdb4-4dc3-9949-eaa9456a2b9d","Type":"ContainerStarted","Data":"67eb8ed920fa51e4ebe62be41e52af9e1ad4708347268e9c612620636fd65377"}
Jul 20 19:57:39.700442 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:39.700049    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 19:57:39.701415 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:39.701383    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xlc4n" event={"ID":"2fbbdeaa-171d-4032-9bd3-7bbd4c009388","Type":"ContainerStarted","Data":"8dba0b704dfc323093c739cac37158b160b703b2bd2339ffa7d68e8bacfcdd4a"}
Jul 20 19:57:39.722322 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:39.722276    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5gtt2" podStartSLOduration=69.910330499 podStartE2EDuration="1m14.722261744s" totalImagesPullingTime="4.811931245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:57:39.721502911 +0000 UTC m=+74.985572958" watchObservedRunningTime="2026-07-20 19:57:39.722261744 +0000 UTC m=+74.986331789"
Jul 20 19:57:39.748735 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:39.748692    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xlc4n" podStartSLOduration=69.925231144 podStartE2EDuration="1m14.748678756s" totalImagesPullingTime="4.823447612s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:57:39.748335175 +0000 UTC m=+75.012405223" watchObservedRunningTime="2026-07-20 19:57:39.748678756 +0000 UTC m=+75.012748802"
Jul 20 19:57:43.775974 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.775920    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jh4d7"]
Jul 20 19:57:43.782319 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.782296    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.786446 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.786425    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 20 19:57:43.786604 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.786579    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 20 19:57:43.787170 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.787144    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 20 19:57:43.787457 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.787436    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 20 19:57:43.787457 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.787455    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 20 19:57:43.787627 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.787613    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hqfdn\""
Jul 20 19:57:43.787704 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.787685    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 20 19:57:43.892711 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.892675    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk8dv\" (UniqueName: \"kubernetes.io/projected/e39ead50-350e-4546-ba14-6e116bb537cb-kube-api-access-kk8dv\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.892924 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.892727    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e39ead50-350e-4546-ba14-6e116bb537cb-node-exporter-textfile\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.892924 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.892756    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e39ead50-350e-4546-ba14-6e116bb537cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.892924 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.892833    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e39ead50-350e-4546-ba14-6e116bb537cb-sys\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.892924 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.892876    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e39ead50-350e-4546-ba14-6e116bb537cb-node-exporter-tls\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.892924 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.892915    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e39ead50-350e-4546-ba14-6e116bb537cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.893191 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.892954    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e39ead50-350e-4546-ba14-6e116bb537cb-metrics-client-ca\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.893191 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.893009    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e39ead50-350e-4546-ba14-6e116bb537cb-node-exporter-wtmp\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.893191 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.893058    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e39ead50-350e-4546-ba14-6e116bb537cb-root\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.994374 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.994346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk8dv\" (UniqueName: \"kubernetes.io/projected/e39ead50-350e-4546-ba14-6e116bb537cb-kube-api-access-kk8dv\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.994575 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.994382    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e39ead50-350e-4546-ba14-6e116bb537cb-node-exporter-textfile\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.994575 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.994400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e39ead50-350e-4546-ba14-6e116bb537cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.994575 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.994423    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e39ead50-350e-4546-ba14-6e116bb537cb-sys\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.994575 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.994443    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e39ead50-350e-4546-ba14-6e116bb537cb-node-exporter-tls\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.994575 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.994461    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e39ead50-350e-4546-ba14-6e116bb537cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.994575 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.994508    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e39ead50-350e-4546-ba14-6e116bb537cb-metrics-client-ca\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.994575 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.994513    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e39ead50-350e-4546-ba14-6e116bb537cb-sys\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.994575 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.994535    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e39ead50-350e-4546-ba14-6e116bb537cb-node-exporter-wtmp\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.994984 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.994590    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e39ead50-350e-4546-ba14-6e116bb537cb-root\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.994984 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.994667    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e39ead50-350e-4546-ba14-6e116bb537cb-root\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.994984 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.994731    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e39ead50-350e-4546-ba14-6e116bb537cb-node-exporter-wtmp\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.994984 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.994753    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e39ead50-350e-4546-ba14-6e116bb537cb-node-exporter-textfile\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.995228 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.995208    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e39ead50-350e-4546-ba14-6e116bb537cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.995294 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.995276    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e39ead50-350e-4546-ba14-6e116bb537cb-metrics-client-ca\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.996946 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.996919    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e39ead50-350e-4546-ba14-6e116bb537cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:43.997109 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:43.997052    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e39ead50-350e-4546-ba14-6e116bb537cb-node-exporter-tls\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:44.010668 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:44.010634    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk8dv\" (UniqueName: \"kubernetes.io/projected/e39ead50-350e-4546-ba14-6e116bb537cb-kube-api-access-kk8dv\") pod \"node-exporter-jh4d7\" (UID: \"e39ead50-350e-4546-ba14-6e116bb537cb\") " pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:44.093355 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:44.093321    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jh4d7"
Jul 20 19:57:44.105760 ip-10-0-143-191 kubenswrapper[2580]: W0720 19:57:44.105727    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode39ead50_350e_4546_ba14_6e116bb537cb.slice/crio-8d40571edf1704df67207a6fce744c7533f4f119debe19c547773029f396281b WatchSource:0}: Error finding container 8d40571edf1704df67207a6fce744c7533f4f119debe19c547773029f396281b: Status 404 returned error can't find the container with id 8d40571edf1704df67207a6fce744c7533f4f119debe19c547773029f396281b
Jul 20 19:57:44.720916 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:44.720871    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jh4d7" event={"ID":"e39ead50-350e-4546-ba14-6e116bb537cb","Type":"ContainerStarted","Data":"8d40571edf1704df67207a6fce744c7533f4f119debe19c547773029f396281b"}
Jul 20 19:57:45.639613 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:45.639588    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:57:45.725491 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:45.725398    2580 generic.go:356] "Generic (PLEG): container finished" podID="e39ead50-350e-4546-ba14-6e116bb537cb" containerID="9e1c3a939a398b39a73c8e75a0c7b36bbe670e78cef66a633a52f6a316103ae4" exitCode=0
Jul 20 19:57:45.725491 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:45.725448    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jh4d7" event={"ID":"e39ead50-350e-4546-ba14-6e116bb537cb","Type":"ContainerDied","Data":"9e1c3a939a398b39a73c8e75a0c7b36bbe670e78cef66a633a52f6a316103ae4"}
Jul 20 19:57:46.731035 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:46.730999    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jh4d7" event={"ID":"e39ead50-350e-4546-ba14-6e116bb537cb","Type":"ContainerStarted","Data":"fa0bfdccc387b3f4903df274d3da45a260d0dad44fb6af9fcdfc387a297febbd"}
Jul 20 19:57:46.731400 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:46.731078    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jh4d7" event={"ID":"e39ead50-350e-4546-ba14-6e116bb537cb","Type":"ContainerStarted","Data":"f16344b58ead41434c8804d0dd63c754aee51211cdc03cc30703dafa19bd75b7"}
Jul 20 19:57:48.643436 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:48.643307    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-77c574ccb-8757d"
Jul 20 19:57:48.669476 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:48.669428    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jh4d7" podStartSLOduration=4.312910518 podStartE2EDuration="5.669409488s" totalImagesPullingTime="1.292434842s" totalInitContainerRuntime="64.064128ms" isStatefulPod=true podCreationTimestamp="2026-07-20 19:57:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:57:46.75635188 +0000 UTC m=+82.020421927" watchObservedRunningTime="2026-07-20 19:57:48.669409488 +0000 UTC m=+83.933479534"
Jul 20 19:57:50.661109 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:50.661054    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw" podUID="e441b5f3-c7c5-4981-bc51-c797d2b43002" containerName="registry" containerID="cri-o://5ec4714ce601e3880d7f727f6b2f4ce9b7c84466a3ff7106c87a5ebf63deef5b" gracePeriod=30
Jul 20 19:57:50.905569 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:50.905542    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:57:51.048193 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.048158    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e441b5f3-c7c5-4981-bc51-c797d2b43002-installation-pull-secrets\") pod \"e441b5f3-c7c5-4981-bc51-c797d2b43002\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") "
Jul 20 19:57:51.048381 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.048211    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-certificates\") pod \"e441b5f3-c7c5-4981-bc51-c797d2b43002\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") "
Jul 20 19:57:51.048381 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.048236    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-tls\") pod \"e441b5f3-c7c5-4981-bc51-c797d2b43002\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") "
Jul 20 19:57:51.048381 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.048280    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e441b5f3-c7c5-4981-bc51-c797d2b43002-ca-trust-extracted\") pod \"e441b5f3-c7c5-4981-bc51-c797d2b43002\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") "
Jul 20 19:57:51.048381 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.048313    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvn4b\" (UniqueName: \"kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-kube-api-access-jvn4b\") pod \"e441b5f3-c7c5-4981-bc51-c797d2b43002\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") "
Jul 20 19:57:51.048381 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.048355    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e441b5f3-c7c5-4981-bc51-c797d2b43002-image-registry-private-configuration\") pod \"e441b5f3-c7c5-4981-bc51-c797d2b43002\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") "
Jul 20 19:57:51.048639 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.048395    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e441b5f3-c7c5-4981-bc51-c797d2b43002-trusted-ca\") pod \"e441b5f3-c7c5-4981-bc51-c797d2b43002\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") "
Jul 20 19:57:51.048639 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.048435    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-bound-sa-token\") pod \"e441b5f3-c7c5-4981-bc51-c797d2b43002\" (UID: \"e441b5f3-c7c5-4981-bc51-c797d2b43002\") "
Jul 20 19:57:51.048746 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.048701    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e441b5f3-c7c5-4981-bc51-c797d2b43002" (UID: "e441b5f3-c7c5-4981-bc51-c797d2b43002"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 19:57:51.049168 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.049134    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e441b5f3-c7c5-4981-bc51-c797d2b43002-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e441b5f3-c7c5-4981-bc51-c797d2b43002" (UID: "e441b5f3-c7c5-4981-bc51-c797d2b43002"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 19:57:51.051236 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.051204    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e441b5f3-c7c5-4981-bc51-c797d2b43002" (UID: "e441b5f3-c7c5-4981-bc51-c797d2b43002"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 19:57:51.051236 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.051229    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e441b5f3-c7c5-4981-bc51-c797d2b43002-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e441b5f3-c7c5-4981-bc51-c797d2b43002" (UID: "e441b5f3-c7c5-4981-bc51-c797d2b43002"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 19:57:51.051478 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.051447    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e441b5f3-c7c5-4981-bc51-c797d2b43002" (UID: "e441b5f3-c7c5-4981-bc51-c797d2b43002"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 19:57:51.051478 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.051462    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e441b5f3-c7c5-4981-bc51-c797d2b43002-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e441b5f3-c7c5-4981-bc51-c797d2b43002" (UID: "e441b5f3-c7c5-4981-bc51-c797d2b43002"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 19:57:51.051607 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.051511    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-kube-api-access-jvn4b" (OuterVolumeSpecName: "kube-api-access-jvn4b") pod "e441b5f3-c7c5-4981-bc51-c797d2b43002" (UID: "e441b5f3-c7c5-4981-bc51-c797d2b43002"). InnerVolumeSpecName "kube-api-access-jvn4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 19:57:51.057211 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.057187    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e441b5f3-c7c5-4981-bc51-c797d2b43002-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e441b5f3-c7c5-4981-bc51-c797d2b43002" (UID: "e441b5f3-c7c5-4981-bc51-c797d2b43002"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 19:57:51.149196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.149163    2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e441b5f3-c7c5-4981-bc51-c797d2b43002-image-registry-private-configuration\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 19:57:51.149196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.149193    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e441b5f3-c7c5-4981-bc51-c797d2b43002-trusted-ca\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 19:57:51.149196 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.149203    2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-bound-sa-token\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 19:57:51.149393 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.149212    2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e441b5f3-c7c5-4981-bc51-c797d2b43002-installation-pull-secrets\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 19:57:51.149393 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.149221    2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-certificates\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 19:57:51.149393 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.149229    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-registry-tls\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 19:57:51.149393 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.149238    2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e441b5f3-c7c5-4981-bc51-c797d2b43002-ca-trust-extracted\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 19:57:51.149393 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.149246    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvn4b\" (UniqueName: \"kubernetes.io/projected/e441b5f3-c7c5-4981-bc51-c797d2b43002-kube-api-access-jvn4b\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 19:57:51.750613 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.750579    2580 generic.go:356] "Generic (PLEG): container finished" podID="e441b5f3-c7c5-4981-bc51-c797d2b43002" containerID="5ec4714ce601e3880d7f727f6b2f4ce9b7c84466a3ff7106c87a5ebf63deef5b" exitCode=0
Jul 20 19:57:51.751107 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.750621    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw" event={"ID":"e441b5f3-c7c5-4981-bc51-c797d2b43002","Type":"ContainerDied","Data":"5ec4714ce601e3880d7f727f6b2f4ce9b7c84466a3ff7106c87a5ebf63deef5b"}
Jul 20 19:57:51.751107 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.750645    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"
Jul 20 19:57:51.751107 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.750667    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b44dfd7b9-5lwkw" event={"ID":"e441b5f3-c7c5-4981-bc51-c797d2b43002","Type":"ContainerDied","Data":"62d7c525a04ecbdeb247d48285c253819b9c101327d46bd890fffa537993273f"}
Jul 20 19:57:51.751107 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.750691    2580 scope.go:117] "RemoveContainer" containerID="5ec4714ce601e3880d7f727f6b2f4ce9b7c84466a3ff7106c87a5ebf63deef5b"
Jul 20 19:57:51.758719 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.758702    2580 scope.go:117] "RemoveContainer" containerID="5ec4714ce601e3880d7f727f6b2f4ce9b7c84466a3ff7106c87a5ebf63deef5b"
Jul 20 19:57:51.758988 ip-10-0-143-191 kubenswrapper[2580]: E0720 19:57:51.758950    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ec4714ce601e3880d7f727f6b2f4ce9b7c84466a3ff7106c87a5ebf63deef5b\": container with ID starting with 5ec4714ce601e3880d7f727f6b2f4ce9b7c84466a3ff7106c87a5ebf63deef5b not found: ID does not exist" containerID="5ec4714ce601e3880d7f727f6b2f4ce9b7c84466a3ff7106c87a5ebf63deef5b"
Jul 20 19:57:51.759041 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.758997    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ec4714ce601e3880d7f727f6b2f4ce9b7c84466a3ff7106c87a5ebf63deef5b"} err="failed to get container status \"5ec4714ce601e3880d7f727f6b2f4ce9b7c84466a3ff7106c87a5ebf63deef5b\": rpc error: code = NotFound desc = could not find container \"5ec4714ce601e3880d7f727f6b2f4ce9b7c84466a3ff7106c87a5ebf63deef5b\": container with ID starting with 5ec4714ce601e3880d7f727f6b2f4ce9b7c84466a3ff7106c87a5ebf63deef5b not found: ID does not exist"
Jul 20 19:57:51.771859 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.771837    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"]
Jul 20 19:57:51.775898 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:51.775879    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5b44dfd7b9-5lwkw"]
Jul 20 19:57:53.174681 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:57:53.174649    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e441b5f3-c7c5-4981-bc51-c797d2b43002" path="/var/lib/kubelet/pods/e441b5f3-c7c5-4981-bc51-c797d2b43002/volumes"
Jul 20 19:58:10.708074 ip-10-0-143-191 kubenswrapper[2580]: I0720 19:58:10.708042    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5gtt2"
Jul 20 20:01:25.036841 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:01:25.036811    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 20:01:25.042250 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:01:25.042229    2580 kubelet.go:1648] "Image garbage collection succeeded"
Jul 20 20:01:25.042634 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:01:25.042617    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 20:02:34.074129 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.074093    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9"]
Jul 20 20:02:34.074535 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.074327    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e441b5f3-c7c5-4981-bc51-c797d2b43002" containerName="registry"
Jul 20 20:02:34.074535 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.074338    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e441b5f3-c7c5-4981-bc51-c797d2b43002" containerName="registry"
Jul 20 20:02:34.074535 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.074379    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e441b5f3-c7c5-4981-bc51-c797d2b43002" containerName="registry"
Jul 20 20:02:34.077263 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.077248    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9"
Jul 20 20:02:34.080992 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.080953    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-w6tt6\""
Jul 20 20:02:34.081219 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.081201    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 20 20:02:34.081304 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.081289    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 20 20:02:34.082994 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.082952    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 20 20:02:34.095084 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.095064    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 20 20:02:34.116228 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.116202    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9"]
Jul 20 20:02:34.192684 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.192645    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65-webhook-cert\") pod \"opendatahub-operator-controller-manager-77cb6d7555-kl2q9\" (UID: \"dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65\") " pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9"
Jul 20 20:02:34.192684 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.192684    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65-tmp\") pod \"opendatahub-operator-controller-manager-77cb6d7555-kl2q9\" (UID: \"dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65\") " pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9"
Jul 20 20:02:34.192919 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.192729    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77cb6d7555-kl2q9\" (UID: \"dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65\") " pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9"
Jul 20 20:02:34.192919 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.192765    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84lzr\" (UniqueName: \"kubernetes.io/projected/dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65-kube-api-access-84lzr\") pod \"opendatahub-operator-controller-manager-77cb6d7555-kl2q9\" (UID: \"dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65\") " pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9"
Jul 20 20:02:34.192919 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.192826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65-manifests\") pod \"opendatahub-operator-controller-manager-77cb6d7555-kl2q9\" (UID: \"dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65\") " pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9"
Jul 20 20:02:34.293388 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.293334    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65-manifests\") pod \"opendatahub-operator-controller-manager-77cb6d7555-kl2q9\" (UID: \"dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65\") " pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9"
Jul 20 20:02:34.293388 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.293404    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65-webhook-cert\") pod \"opendatahub-operator-controller-manager-77cb6d7555-kl2q9\" (UID: \"dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65\") " pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9"
Jul 20 20:02:34.293660 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.293424    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65-tmp\") pod \"opendatahub-operator-controller-manager-77cb6d7555-kl2q9\" (UID: \"dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65\") " pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9"
Jul 20 20:02:34.293660 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.293448    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77cb6d7555-kl2q9\" (UID: \"dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65\") " pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9"
Jul 20 20:02:34.293660 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.293464    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84lzr\" (UniqueName: \"kubernetes.io/projected/dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65-kube-api-access-84lzr\") pod \"opendatahub-operator-controller-manager-77cb6d7555-kl2q9\" (UID: \"dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65\") " pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9"
Jul 20 20:02:34.293820 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.293732    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65-manifests\") pod \"opendatahub-operator-controller-manager-77cb6d7555-kl2q9\" (UID: \"dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65\") " pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9"
Jul 20 20:02:34.295775 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.295741    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65-tmp\") pod \"opendatahub-operator-controller-manager-77cb6d7555-kl2q9\" (UID: \"dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65\") " pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9"
Jul 20 20:02:34.296013 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.295995    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77cb6d7555-kl2q9\" (UID: \"dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65\") " pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9"
Jul 20 20:02:34.296058 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.296023    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65-webhook-cert\") pod \"opendatahub-operator-controller-manager-77cb6d7555-kl2q9\" (UID: \"dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65\") " pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9"
Jul 20 20:02:34.304291 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.304263    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84lzr\" (UniqueName: \"kubernetes.io/projected/dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65-kube-api-access-84lzr\") pod \"opendatahub-operator-controller-manager-77cb6d7555-kl2q9\" (UID: \"dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65\") " pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9"
Jul 20 20:02:34.386294 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.386187    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9"
Jul 20 20:02:34.526557 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.526523    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9"]
Jul 20 20:02:34.529809 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:02:34.529786    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbc0ed85_28a8_4278_ae9b_0c2da1ce0c65.slice/crio-4cf285db351651f0200b827014f938166827cd54a3a7809392f083e4d8119ef3 WatchSource:0}: Error finding container 4cf285db351651f0200b827014f938166827cd54a3a7809392f083e4d8119ef3: Status 404 returned error can't find the container with id 4cf285db351651f0200b827014f938166827cd54a3a7809392f083e4d8119ef3
Jul 20 20:02:34.531638 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.531620    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 20:02:34.657583 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:34.657483    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9" event={"ID":"dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65","Type":"ContainerStarted","Data":"4cf285db351651f0200b827014f938166827cd54a3a7809392f083e4d8119ef3"}
Jul 20 20:02:38.675764 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:38.675728    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9" event={"ID":"dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65","Type":"ContainerStarted","Data":"4bec667bc3235d3a9bc94a93b346e120f5d369e744cad43dba0edddea8b670e4"}
Jul 20 20:02:39.680004 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:39.679953    2580 generic.go:356] "Generic (PLEG): container finished" podID="dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65" containerID="4bec667bc3235d3a9bc94a93b346e120f5d369e744cad43dba0edddea8b670e4" exitCode=0
Jul 20 20:02:39.680374 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:39.680022    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9" event={"ID":"dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65","Type":"ContainerDied","Data":"4bec667bc3235d3a9bc94a93b346e120f5d369e744cad43dba0edddea8b670e4"}
Jul 20 20:02:40.685317 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:40.685281    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9" event={"ID":"dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65","Type":"ContainerStarted","Data":"8c9f38600e1c678cfd594963dfb0b992932a8d95c59ba681d3f8fbd4dfb02ed4"}
Jul 20 20:02:40.685799 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:40.685415    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9"
Jul 20 20:02:40.718541 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:40.718496    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9" podStartSLOduration=2.102822889 podStartE2EDuration="6.718482318s" totalImagesPullingTime="3.539297888s" totalInitContainerRuntime="1.076361541s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:02:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:02:40.716526665 +0000 UTC m=+375.980596714" watchObservedRunningTime="2026-07-20 20:02:40.718482318 +0000 UTC m=+375.982552419"
Jul 20 20:02:51.691242 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:51.691210    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-77cb6d7555-kl2q9"
Jul 20 20:02:54.101937 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.101905    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-66fb5dc8ff-pmhvf"]
Jul 20 20:02:54.107704 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.107684    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66fb5dc8ff-pmhvf"
Jul 20 20:02:54.112512 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.112487    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 20 20:02:54.112669 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.112535    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 20 20:02:54.114176 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.113982    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8tkln\""
Jul 20 20:02:54.114176 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.113996    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 20 20:02:54.114322 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.114301    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 20 20:02:54.129942 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.129921    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66fb5dc8ff-pmhvf"]
Jul 20 20:02:54.210674 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.210637    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hkvfk"]
Jul 20 20:02:54.213503 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.213481    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hkvfk"
Jul 20 20:02:54.216972 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.216939    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 20 20:02:54.217072 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.216948    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-hzp8l\""
Jul 20 20:02:54.234908 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.234883    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hkvfk"]
Jul 20 20:02:54.240842 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.240820    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55jmr\" (UniqueName: \"kubernetes.io/projected/db75232b-3ac5-48cc-9aea-3b1ec3c1c551-kube-api-access-55jmr\") pod \"kube-auth-proxy-66fb5dc8ff-pmhvf\" (UID: \"db75232b-3ac5-48cc-9aea-3b1ec3c1c551\") " pod="openshift-ingress/kube-auth-proxy-66fb5dc8ff-pmhvf"
Jul 20 20:02:54.240949 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.240870    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db75232b-3ac5-48cc-9aea-3b1ec3c1c551-tls-certs\") pod \"kube-auth-proxy-66fb5dc8ff-pmhvf\" (UID: \"db75232b-3ac5-48cc-9aea-3b1ec3c1c551\") " pod="openshift-ingress/kube-auth-proxy-66fb5dc8ff-pmhvf"
Jul 20 20:02:54.241036 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.241011    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db75232b-3ac5-48cc-9aea-3b1ec3c1c551-tmp\") pod \"kube-auth-proxy-66fb5dc8ff-pmhvf\" (UID: \"db75232b-3ac5-48cc-9aea-3b1ec3c1c551\") " pod="openshift-ingress/kube-auth-proxy-66fb5dc8ff-pmhvf"
Jul 20 20:02:54.342265 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.342226    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db75232b-3ac5-48cc-9aea-3b1ec3c1c551-tmp\") pod \"kube-auth-proxy-66fb5dc8ff-pmhvf\" (UID: \"db75232b-3ac5-48cc-9aea-3b1ec3c1c551\") " pod="openshift-ingress/kube-auth-proxy-66fb5dc8ff-pmhvf"
Jul 20 20:02:54.342454 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.342291    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqn56\" (UniqueName: \"kubernetes.io/projected/2c339348-9029-4d9b-854b-a88e192e3266-kube-api-access-mqn56\") pod \"odh-model-controller-7998c67dc4-hkvfk\" (UID: \"2c339348-9029-4d9b-854b-a88e192e3266\") " pod="opendatahub/odh-model-controller-7998c67dc4-hkvfk"
Jul 20 20:02:54.342454 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.342322    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55jmr\" (UniqueName: \"kubernetes.io/projected/db75232b-3ac5-48cc-9aea-3b1ec3c1c551-kube-api-access-55jmr\") pod \"kube-auth-proxy-66fb5dc8ff-pmhvf\" (UID: \"db75232b-3ac5-48cc-9aea-3b1ec3c1c551\") " pod="openshift-ingress/kube-auth-proxy-66fb5dc8ff-pmhvf"
Jul 20 20:02:54.342454 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.342358    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db75232b-3ac5-48cc-9aea-3b1ec3c1c551-tls-certs\") pod \"kube-auth-proxy-66fb5dc8ff-pmhvf\" (UID: \"db75232b-3ac5-48cc-9aea-3b1ec3c1c551\") " pod="openshift-ingress/kube-auth-proxy-66fb5dc8ff-pmhvf"
Jul 20 20:02:54.342454 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.342385    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c339348-9029-4d9b-854b-a88e192e3266-cert\") pod \"odh-model-controller-7998c67dc4-hkvfk\" (UID: \"2c339348-9029-4d9b-854b-a88e192e3266\") " pod="opendatahub/odh-model-controller-7998c67dc4-hkvfk"
Jul 20 20:02:54.344416 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.344394    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db75232b-3ac5-48cc-9aea-3b1ec3c1c551-tmp\") pod \"kube-auth-proxy-66fb5dc8ff-pmhvf\" (UID: \"db75232b-3ac5-48cc-9aea-3b1ec3c1c551\") " pod="openshift-ingress/kube-auth-proxy-66fb5dc8ff-pmhvf"
Jul 20 20:02:54.344705 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.344687    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db75232b-3ac5-48cc-9aea-3b1ec3c1c551-tls-certs\") pod \"kube-auth-proxy-66fb5dc8ff-pmhvf\" (UID: \"db75232b-3ac5-48cc-9aea-3b1ec3c1c551\") " pod="openshift-ingress/kube-auth-proxy-66fb5dc8ff-pmhvf"
Jul 20 20:02:54.352321 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.352246    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55jmr\" (UniqueName: \"kubernetes.io/projected/db75232b-3ac5-48cc-9aea-3b1ec3c1c551-kube-api-access-55jmr\") pod \"kube-auth-proxy-66fb5dc8ff-pmhvf\" (UID: \"db75232b-3ac5-48cc-9aea-3b1ec3c1c551\") " pod="openshift-ingress/kube-auth-proxy-66fb5dc8ff-pmhvf"
Jul 20 20:02:54.417539 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.417504    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66fb5dc8ff-pmhvf"
Jul 20 20:02:54.442703 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.442670    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c339348-9029-4d9b-854b-a88e192e3266-cert\") pod \"odh-model-controller-7998c67dc4-hkvfk\" (UID: \"2c339348-9029-4d9b-854b-a88e192e3266\") " pod="opendatahub/odh-model-controller-7998c67dc4-hkvfk"
Jul 20 20:02:54.442840 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.442731    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqn56\" (UniqueName: \"kubernetes.io/projected/2c339348-9029-4d9b-854b-a88e192e3266-kube-api-access-mqn56\") pod \"odh-model-controller-7998c67dc4-hkvfk\" (UID: \"2c339348-9029-4d9b-854b-a88e192e3266\") " pod="opendatahub/odh-model-controller-7998c67dc4-hkvfk"
Jul 20 20:02:54.445346 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.445320    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c339348-9029-4d9b-854b-a88e192e3266-cert\") pod \"odh-model-controller-7998c67dc4-hkvfk\" (UID: \"2c339348-9029-4d9b-854b-a88e192e3266\") " pod="opendatahub/odh-model-controller-7998c67dc4-hkvfk"
Jul 20 20:02:54.456695 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.456666    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqn56\" (UniqueName: \"kubernetes.io/projected/2c339348-9029-4d9b-854b-a88e192e3266-kube-api-access-mqn56\") pod \"odh-model-controller-7998c67dc4-hkvfk\" (UID: \"2c339348-9029-4d9b-854b-a88e192e3266\") " pod="opendatahub/odh-model-controller-7998c67dc4-hkvfk"
Jul 20 20:02:54.523108 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.523080    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hkvfk"
Jul 20 20:02:54.546688 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.546665    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66fb5dc8ff-pmhvf"]
Jul 20 20:02:54.549286 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:02:54.549259    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb75232b_3ac5_48cc_9aea_3b1ec3c1c551.slice/crio-3ac2bf03b519f4cf4b820857a577c6828ce7dde95421f0499273206633062565 WatchSource:0}: Error finding container 3ac2bf03b519f4cf4b820857a577c6828ce7dde95421f0499273206633062565: Status 404 returned error can't find the container with id 3ac2bf03b519f4cf4b820857a577c6828ce7dde95421f0499273206633062565
Jul 20 20:02:54.651350 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.651276    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hkvfk"]
Jul 20 20:02:54.654069 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:02:54.654033    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c339348_9029_4d9b_854b_a88e192e3266.slice/crio-e9236869c8f77e9d23e57f877fef4af5330b6aaf301be81303b648accc516fd1 WatchSource:0}: Error finding container e9236869c8f77e9d23e57f877fef4af5330b6aaf301be81303b648accc516fd1: Status 404 returned error can't find the container with id e9236869c8f77e9d23e57f877fef4af5330b6aaf301be81303b648accc516fd1
Jul 20 20:02:54.732859 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.732823    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hkvfk" event={"ID":"2c339348-9029-4d9b-854b-a88e192e3266","Type":"ContainerStarted","Data":"e9236869c8f77e9d23e57f877fef4af5330b6aaf301be81303b648accc516fd1"}
Jul 20 20:02:54.733863 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:54.733836    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66fb5dc8ff-pmhvf" event={"ID":"db75232b-3ac5-48cc-9aea-3b1ec3c1c551","Type":"ContainerStarted","Data":"3ac2bf03b519f4cf4b820857a577c6828ce7dde95421f0499273206633062565"}
Jul 20 20:02:55.522080 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:55.522047    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f6758cc7d-zs24s"]
Jul 20 20:02:55.525287 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:55.525257    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7f6758cc7d-zs24s"
Jul 20 20:02:55.533016 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:55.532840    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 20 20:02:55.533268 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:55.533182    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 20 20:02:55.534751 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:55.534544    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 20 20:02:55.534822 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:55.534785    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7vkk8\""
Jul 20 20:02:55.535022 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:55.535002    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 20 20:02:55.535318 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:55.535149    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 20 20:02:55.646743 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:55.646660    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f6758cc7d-zs24s"]
Jul 20 20:02:55.652128 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:55.652100    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a4c6b53a-c84f-4531-a852-f827f8a38a46-manager-config\") pod \"lws-controller-manager-7f6758cc7d-zs24s\" (UID: \"a4c6b53a-c84f-4531-a852-f827f8a38a46\") " pod="openshift-lws-operator/lws-controller-manager-7f6758cc7d-zs24s"
Jul 20 20:02:55.652294 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:55.652165    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a4c6b53a-c84f-4531-a852-f827f8a38a46-cert\") pod \"lws-controller-manager-7f6758cc7d-zs24s\" (UID: \"a4c6b53a-c84f-4531-a852-f827f8a38a46\") " pod="openshift-lws-operator/lws-controller-manager-7f6758cc7d-zs24s"
Jul 20 20:02:55.652294 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:55.652251    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a4c6b53a-c84f-4531-a852-f827f8a38a46-metrics-cert\") pod \"lws-controller-manager-7f6758cc7d-zs24s\" (UID: \"a4c6b53a-c84f-4531-a852-f827f8a38a46\") " pod="openshift-lws-operator/lws-controller-manager-7f6758cc7d-zs24s"
Jul 20 20:02:55.652414 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:55.652338    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b59kx\" (UniqueName: \"kubernetes.io/projected/a4c6b53a-c84f-4531-a852-f827f8a38a46-kube-api-access-b59kx\") pod \"lws-controller-manager-7f6758cc7d-zs24s\" (UID: \"a4c6b53a-c84f-4531-a852-f827f8a38a46\") " pod="openshift-lws-operator/lws-controller-manager-7f6758cc7d-zs24s"
Jul 20 20:02:55.753063 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:55.753026    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a4c6b53a-c84f-4531-a852-f827f8a38a46-cert\") pod \"lws-controller-manager-7f6758cc7d-zs24s\" (UID: \"a4c6b53a-c84f-4531-a852-f827f8a38a46\") " pod="openshift-lws-operator/lws-controller-manager-7f6758cc7d-zs24s"
Jul 20 20:02:55.753254 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:55.753077    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a4c6b53a-c84f-4531-a852-f827f8a38a46-metrics-cert\") pod \"lws-controller-manager-7f6758cc7d-zs24s\" (UID: \"a4c6b53a-c84f-4531-a852-f827f8a38a46\") " pod="openshift-lws-operator/lws-controller-manager-7f6758cc7d-zs24s"
Jul 20 20:02:55.753254 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:55.753147    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b59kx\" (UniqueName: \"kubernetes.io/projected/a4c6b53a-c84f-4531-a852-f827f8a38a46-kube-api-access-b59kx\") pod \"lws-controller-manager-7f6758cc7d-zs24s\" (UID: \"a4c6b53a-c84f-4531-a852-f827f8a38a46\") " pod="openshift-lws-operator/lws-controller-manager-7f6758cc7d-zs24s"
Jul 20 20:02:55.753254 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:55.753184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a4c6b53a-c84f-4531-a852-f827f8a38a46-manager-config\") pod \"lws-controller-manager-7f6758cc7d-zs24s\" (UID: \"a4c6b53a-c84f-4531-a852-f827f8a38a46\") " pod="openshift-lws-operator/lws-controller-manager-7f6758cc7d-zs24s"
Jul 20 20:02:55.753945 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:55.753919    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a4c6b53a-c84f-4531-a852-f827f8a38a46-manager-config\") pod \"lws-controller-manager-7f6758cc7d-zs24s\" (UID: \"a4c6b53a-c84f-4531-a852-f827f8a38a46\") " pod="openshift-lws-operator/lws-controller-manager-7f6758cc7d-zs24s"
Jul 20 20:02:55.756298 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:55.756273    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a4c6b53a-c84f-4531-a852-f827f8a38a46-cert\") pod \"lws-controller-manager-7f6758cc7d-zs24s\" (UID: \"a4c6b53a-c84f-4531-a852-f827f8a38a46\") " pod="openshift-lws-operator/lws-controller-manager-7f6758cc7d-zs24s"
Jul 20 20:02:55.756528 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:55.756510    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a4c6b53a-c84f-4531-a852-f827f8a38a46-metrics-cert\") pod \"lws-controller-manager-7f6758cc7d-zs24s\" (UID: \"a4c6b53a-c84f-4531-a852-f827f8a38a46\") " pod="openshift-lws-operator/lws-controller-manager-7f6758cc7d-zs24s"
Jul 20 20:02:55.779156 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:55.779085    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b59kx\" (UniqueName: \"kubernetes.io/projected/a4c6b53a-c84f-4531-a852-f827f8a38a46-kube-api-access-b59kx\") pod \"lws-controller-manager-7f6758cc7d-zs24s\" (UID: \"a4c6b53a-c84f-4531-a852-f827f8a38a46\") " pod="openshift-lws-operator/lws-controller-manager-7f6758cc7d-zs24s"
Jul 20 20:02:55.838787 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:55.838751    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7f6758cc7d-zs24s"
Jul 20 20:02:56.011247 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:56.011213    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f6758cc7d-zs24s"]
Jul 20 20:02:56.014333 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:02:56.014299    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4c6b53a_c84f_4531_a852_f827f8a38a46.slice/crio-f5812dde6c3bf7bef7c07079d94bb0d1c108a25abcd4db169e17888facfaa93f WatchSource:0}: Error finding container f5812dde6c3bf7bef7c07079d94bb0d1c108a25abcd4db169e17888facfaa93f: Status 404 returned error can't find the container with id f5812dde6c3bf7bef7c07079d94bb0d1c108a25abcd4db169e17888facfaa93f
Jul 20 20:02:56.744088 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:56.744049    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7f6758cc7d-zs24s" event={"ID":"a4c6b53a-c84f-4531-a852-f827f8a38a46","Type":"ContainerStarted","Data":"f5812dde6c3bf7bef7c07079d94bb0d1c108a25abcd4db169e17888facfaa93f"}
Jul 20 20:02:58.753889 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:58.753735    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hkvfk" event={"ID":"2c339348-9029-4d9b-854b-a88e192e3266","Type":"ContainerStarted","Data":"660332eed8d81bace15d9a6af50651ac1af10f07858b8012c7bfa150bdfe2b2d"}
Jul 20 20:02:58.754386 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:58.754015    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-hkvfk"
Jul 20 20:02:58.755364 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:58.755333    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66fb5dc8ff-pmhvf" event={"ID":"db75232b-3ac5-48cc-9aea-3b1ec3c1c551","Type":"ContainerStarted","Data":"fc962f2561605a5589a660efb7c41f3f2b6fa115eca82dab9a484bd9da4634e0"}
Jul 20 20:02:58.770670 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:02:58.770612    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-hkvfk" podStartSLOduration=1.07367651 podStartE2EDuration="4.770571037s" totalImagesPullingTime="3.696894527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:02:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:02:58.770342211 +0000 UTC m=+394.034412280" watchObservedRunningTime="2026-07-20 20:02:58.770571037 +0000 UTC m=+394.034641083"
Jul 20 20:03:00.765053 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:03:00.765015    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7f6758cc7d-zs24s" event={"ID":"a4c6b53a-c84f-4531-a852-f827f8a38a46","Type":"ContainerStarted","Data":"2bb1a59b7df0068795b62a8f0013bfaf6875bf8069268dc9187944d6c32e26d0"}
Jul 20 20:03:00.765501 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:03:00.765087    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7f6758cc7d-zs24s"
Jul 20 20:03:00.787003 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:03:00.786934    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-66fb5dc8ff-pmhvf" podStartSLOduration=2.984752915 podStartE2EDuration="6.786917666s" totalImagesPullingTime="3.802164751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:02:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:02:58.790150553 +0000 UTC m=+394.054220600" watchObservedRunningTime="2026-07-20 20:03:00.786917666 +0000 UTC m=+396.050987712"
Jul 20 20:03:00.787151 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:03:00.787091    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7f6758cc7d-zs24s" podStartSLOduration=1.742690923 podStartE2EDuration="5.787084456s" totalImagesPullingTime="4.044393533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:02:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:03:00.785536546 +0000 UTC m=+396.049606592" watchObservedRunningTime="2026-07-20 20:03:00.787084456 +0000 UTC m=+396.051154501"
Jul 20 20:03:09.761931 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:03:09.761899    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-hkvfk"
Jul 20 20:03:11.770940 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:03:11.770910    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7f6758cc7d-zs24s"
Jul 20 20:04:51.612273 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:04:51.612236    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2pkv2"]
Jul 20 20:04:51.615287 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:04:51.615271    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pkv2"
Jul 20 20:04:51.618679 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:04:51.618658    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 20 20:04:51.618849 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:04:51.618670    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 20 20:04:51.619270 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:04:51.619249    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 20 20:04:51.619395 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:04:51.619278    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9q6r4\""
Jul 20 20:04:51.619395 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:04:51.619322    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 20 20:04:51.625082 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:04:51.625061    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2pkv2"]
Jul 20 20:04:51.655013 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:04:51.654986    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bdf41e73-0a8c-4200-a3d9-5aa7afac963b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2pkv2\" (UID: \"bdf41e73-0a8c-4200-a3d9-5aa7afac963b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pkv2"
Jul 20 20:04:51.655128 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:04:51.655020    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grq6c\" (UniqueName: \"kubernetes.io/projected/bdf41e73-0a8c-4200-a3d9-5aa7afac963b-kube-api-access-grq6c\") pod \"kuadrant-console-plugin-66b5774985-2pkv2\" (UID: \"bdf41e73-0a8c-4200-a3d9-5aa7afac963b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pkv2"
Jul 20 20:04:51.655128 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:04:51.655077    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bdf41e73-0a8c-4200-a3d9-5aa7afac963b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2pkv2\" (UID: \"bdf41e73-0a8c-4200-a3d9-5aa7afac963b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pkv2"
Jul 20 20:04:51.755511 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:04:51.755482    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grq6c\" (UniqueName: \"kubernetes.io/projected/bdf41e73-0a8c-4200-a3d9-5aa7afac963b-kube-api-access-grq6c\") pod \"kuadrant-console-plugin-66b5774985-2pkv2\" (UID: \"bdf41e73-0a8c-4200-a3d9-5aa7afac963b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pkv2"
Jul 20 20:04:51.755683 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:04:51.755546    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bdf41e73-0a8c-4200-a3d9-5aa7afac963b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2pkv2\" (UID: \"bdf41e73-0a8c-4200-a3d9-5aa7afac963b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pkv2"
Jul 20 20:04:51.755683 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:04:51.755604    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bdf41e73-0a8c-4200-a3d9-5aa7afac963b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2pkv2\" (UID: \"bdf41e73-0a8c-4200-a3d9-5aa7afac963b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pkv2"
Jul 20 20:04:51.756204 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:04:51.756185    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bdf41e73-0a8c-4200-a3d9-5aa7afac963b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2pkv2\" (UID: \"bdf41e73-0a8c-4200-a3d9-5aa7afac963b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pkv2"
Jul 20 20:04:51.758096 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:04:51.758075    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bdf41e73-0a8c-4200-a3d9-5aa7afac963b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2pkv2\" (UID: \"bdf41e73-0a8c-4200-a3d9-5aa7afac963b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pkv2"
Jul 20 20:04:51.765656 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:04:51.765625    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grq6c\" (UniqueName: \"kubernetes.io/projected/bdf41e73-0a8c-4200-a3d9-5aa7afac963b-kube-api-access-grq6c\") pod \"kuadrant-console-plugin-66b5774985-2pkv2\" (UID: \"bdf41e73-0a8c-4200-a3d9-5aa7afac963b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pkv2"
Jul 20 20:04:51.926221 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:04:51.926132    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pkv2"
Jul 20 20:04:52.054448 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:04:52.054417    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2pkv2"]
Jul 20 20:04:52.057695 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:04:52.057666    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbdf41e73_0a8c_4200_a3d9_5aa7afac963b.slice/crio-05f554eb070e6ba88b4be4d178eabd23fe72ef6350a8763cd40dd460ab84fff6 WatchSource:0}: Error finding container 05f554eb070e6ba88b4be4d178eabd23fe72ef6350a8763cd40dd460ab84fff6: Status 404 returned error can't find the container with id 05f554eb070e6ba88b4be4d178eabd23fe72ef6350a8763cd40dd460ab84fff6
Jul 20 20:04:52.166183 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:04:52.166150    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pkv2" event={"ID":"bdf41e73-0a8c-4200-a3d9-5aa7afac963b","Type":"ContainerStarted","Data":"05f554eb070e6ba88b4be4d178eabd23fe72ef6350a8763cd40dd460ab84fff6"}
Jul 20 20:05:16.267781 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:16.267749    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pkv2" event={"ID":"bdf41e73-0a8c-4200-a3d9-5aa7afac963b","Type":"ContainerStarted","Data":"60bfa89427b78a8e2f4eaa53b2c555417a61894f42a37dff369e2ba858a179f9"}
Jul 20 20:05:16.285940 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:16.285886    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pkv2" podStartSLOduration=1.252389102 podStartE2EDuration="25.28586852s" totalImagesPullingTime="24.033479418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:04:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:05:16.28576601 +0000 UTC m=+531.549836056" watchObservedRunningTime="2026-07-20 20:05:16.28586852 +0000 UTC m=+531.549938568"
Jul 20 20:05:47.001880 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:47.001846    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-n4zhv"]
Jul 20 20:05:47.066140 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:47.066055    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-n4zhv"]
Jul 20 20:05:47.066140 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:47.066137    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-n4zhv"
Jul 20 20:05:47.069936 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:47.069917    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-54dlw\""
Jul 20 20:05:47.117020 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:47.116980    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxlrr\" (UniqueName: \"kubernetes.io/projected/4939d813-2c85-4615-8425-bb61aa96f5cb-kube-api-access-pxlrr\") pod \"authorino-58cd955675-n4zhv\" (UID: \"4939d813-2c85-4615-8425-bb61aa96f5cb\") " pod="kuadrant-system/authorino-58cd955675-n4zhv"
Jul 20 20:05:47.217488 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:47.217450    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxlrr\" (UniqueName: \"kubernetes.io/projected/4939d813-2c85-4615-8425-bb61aa96f5cb-kube-api-access-pxlrr\") pod \"authorino-58cd955675-n4zhv\" (UID: \"4939d813-2c85-4615-8425-bb61aa96f5cb\") " pod="kuadrant-system/authorino-58cd955675-n4zhv"
Jul 20 20:05:47.226649 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:47.226622    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxlrr\" (UniqueName: \"kubernetes.io/projected/4939d813-2c85-4615-8425-bb61aa96f5cb-kube-api-access-pxlrr\") pod \"authorino-58cd955675-n4zhv\" (UID: \"4939d813-2c85-4615-8425-bb61aa96f5cb\") " pod="kuadrant-system/authorino-58cd955675-n4zhv"
Jul 20 20:05:47.256890 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:47.256801    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-7nxbt"]
Jul 20 20:05:47.260146 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:47.260125    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7nxbt"
Jul 20 20:05:47.267570 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:47.267552    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7nxbt"]
Jul 20 20:05:47.318360 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:47.318327    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxgzs\" (UniqueName: \"kubernetes.io/projected/9a41f580-2b76-45a6-a072-953c7c2e1317-kube-api-access-pxgzs\") pod \"authorino-75d556bd77-7nxbt\" (UID: \"9a41f580-2b76-45a6-a072-953c7c2e1317\") " pod="kuadrant-system/authorino-75d556bd77-7nxbt"
Jul 20 20:05:47.375394 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:47.375364    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-n4zhv"
Jul 20 20:05:47.419274 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:47.419237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxgzs\" (UniqueName: \"kubernetes.io/projected/9a41f580-2b76-45a6-a072-953c7c2e1317-kube-api-access-pxgzs\") pod \"authorino-75d556bd77-7nxbt\" (UID: \"9a41f580-2b76-45a6-a072-953c7c2e1317\") " pod="kuadrant-system/authorino-75d556bd77-7nxbt"
Jul 20 20:05:47.427789 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:47.427757    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxgzs\" (UniqueName: \"kubernetes.io/projected/9a41f580-2b76-45a6-a072-953c7c2e1317-kube-api-access-pxgzs\") pod \"authorino-75d556bd77-7nxbt\" (UID: \"9a41f580-2b76-45a6-a072-953c7c2e1317\") " pod="kuadrant-system/authorino-75d556bd77-7nxbt"
Jul 20 20:05:47.502940 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:47.502915    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-n4zhv"]
Jul 20 20:05:47.505683 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:05:47.505650    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4939d813_2c85_4615_8425_bb61aa96f5cb.slice/crio-e9fbd47c67cb29a9f0e77ad30f726bacf668ccc381b2e9cf7c0114b7bdfd6a3b WatchSource:0}: Error finding container e9fbd47c67cb29a9f0e77ad30f726bacf668ccc381b2e9cf7c0114b7bdfd6a3b: Status 404 returned error can't find the container with id e9fbd47c67cb29a9f0e77ad30f726bacf668ccc381b2e9cf7c0114b7bdfd6a3b
Jul 20 20:05:47.570257 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:47.570220    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7nxbt"
Jul 20 20:05:47.692773 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:47.692750    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7nxbt"]
Jul 20 20:05:47.695147 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:05:47.695123    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a41f580_2b76_45a6_a072_953c7c2e1317.slice/crio-b5e31fde5ef3809b499b27de2c4f9eb1141e7d6ac7b4593fca1d2e812c4ea3a3 WatchSource:0}: Error finding container b5e31fde5ef3809b499b27de2c4f9eb1141e7d6ac7b4593fca1d2e812c4ea3a3: Status 404 returned error can't find the container with id b5e31fde5ef3809b499b27de2c4f9eb1141e7d6ac7b4593fca1d2e812c4ea3a3
Jul 20 20:05:48.386912 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:48.386844    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-n4zhv" event={"ID":"4939d813-2c85-4615-8425-bb61aa96f5cb","Type":"ContainerStarted","Data":"e9fbd47c67cb29a9f0e77ad30f726bacf668ccc381b2e9cf7c0114b7bdfd6a3b"}
Jul 20 20:05:48.388912 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:48.388874    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7nxbt" event={"ID":"9a41f580-2b76-45a6-a072-953c7c2e1317","Type":"ContainerStarted","Data":"b5e31fde5ef3809b499b27de2c4f9eb1141e7d6ac7b4593fca1d2e812c4ea3a3"}
Jul 20 20:05:51.403310 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:51.403266    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-n4zhv" event={"ID":"4939d813-2c85-4615-8425-bb61aa96f5cb","Type":"ContainerStarted","Data":"e6d10c7fa2a095839d70d6e147f0dcd7e2a67bee2257c8e77de819cd99509eb8"}
Jul 20 20:05:51.404608 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:51.404582    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7nxbt" event={"ID":"9a41f580-2b76-45a6-a072-953c7c2e1317","Type":"ContainerStarted","Data":"6bac81e4d29421d18a347d0a09f130ea3681029b8f43b47d6e708516cf66833e"}
Jul 20 20:05:51.419355 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:51.419312    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-n4zhv" podStartSLOduration=1.947168397 podStartE2EDuration="5.419299013s" totalImagesPullingTime="3.472130616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:05:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:05:51.418536934 +0000 UTC m=+566.682606979" watchObservedRunningTime="2026-07-20 20:05:51.419299013 +0000 UTC m=+566.683369059"
Jul 20 20:05:51.433784 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:51.433742    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-7nxbt" podStartSLOduration=1.153716032 podStartE2EDuration="4.433728073s" totalImagesPullingTime="3.280012041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:05:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:05:51.433234699 +0000 UTC m=+566.697304737" watchObservedRunningTime="2026-07-20 20:05:51.433728073 +0000 UTC m=+566.697798119"
Jul 20 20:05:51.457694 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:51.457664    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-n4zhv"]
Jul 20 20:05:53.412310 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:53.412251    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-n4zhv" podUID="4939d813-2c85-4615-8425-bb61aa96f5cb" containerName="authorino" containerID="cri-o://e6d10c7fa2a095839d70d6e147f0dcd7e2a67bee2257c8e77de819cd99509eb8" gracePeriod=30
Jul 20 20:05:53.645280 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:53.645253    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-n4zhv"
Jul 20 20:05:53.667811 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:53.667765    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxlrr\" (UniqueName: \"kubernetes.io/projected/4939d813-2c85-4615-8425-bb61aa96f5cb-kube-api-access-pxlrr\") pod \"4939d813-2c85-4615-8425-bb61aa96f5cb\" (UID: \"4939d813-2c85-4615-8425-bb61aa96f5cb\") "
Jul 20 20:05:53.669738 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:53.669710    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4939d813-2c85-4615-8425-bb61aa96f5cb-kube-api-access-pxlrr" (OuterVolumeSpecName: "kube-api-access-pxlrr") pod "4939d813-2c85-4615-8425-bb61aa96f5cb" (UID: "4939d813-2c85-4615-8425-bb61aa96f5cb"). InnerVolumeSpecName "kube-api-access-pxlrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:05:53.769275 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:53.769233    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxlrr\" (UniqueName: \"kubernetes.io/projected/4939d813-2c85-4615-8425-bb61aa96f5cb-kube-api-access-pxlrr\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:05:54.417165 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:54.417128    2580 generic.go:356] "Generic (PLEG): container finished" podID="4939d813-2c85-4615-8425-bb61aa96f5cb" containerID="e6d10c7fa2a095839d70d6e147f0dcd7e2a67bee2257c8e77de819cd99509eb8" exitCode=0
Jul 20 20:05:54.417563 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:54.417169    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-n4zhv"
Jul 20 20:05:54.417563 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:54.417201    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-n4zhv" event={"ID":"4939d813-2c85-4615-8425-bb61aa96f5cb","Type":"ContainerDied","Data":"e6d10c7fa2a095839d70d6e147f0dcd7e2a67bee2257c8e77de819cd99509eb8"}
Jul 20 20:05:54.417563 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:54.417232    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-n4zhv" event={"ID":"4939d813-2c85-4615-8425-bb61aa96f5cb","Type":"ContainerDied","Data":"e9fbd47c67cb29a9f0e77ad30f726bacf668ccc381b2e9cf7c0114b7bdfd6a3b"}
Jul 20 20:05:54.417563 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:54.417248    2580 scope.go:117] "RemoveContainer" containerID="e6d10c7fa2a095839d70d6e147f0dcd7e2a67bee2257c8e77de819cd99509eb8"
Jul 20 20:05:54.425020 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:54.425001    2580 scope.go:117] "RemoveContainer" containerID="e6d10c7fa2a095839d70d6e147f0dcd7e2a67bee2257c8e77de819cd99509eb8"
Jul 20 20:05:54.425266 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:05:54.425247    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6d10c7fa2a095839d70d6e147f0dcd7e2a67bee2257c8e77de819cd99509eb8\": container with ID starting with e6d10c7fa2a095839d70d6e147f0dcd7e2a67bee2257c8e77de819cd99509eb8 not found: ID does not exist" containerID="e6d10c7fa2a095839d70d6e147f0dcd7e2a67bee2257c8e77de819cd99509eb8"
Jul 20 20:05:54.425330 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:54.425274    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6d10c7fa2a095839d70d6e147f0dcd7e2a67bee2257c8e77de819cd99509eb8"} err="failed to get container status \"e6d10c7fa2a095839d70d6e147f0dcd7e2a67bee2257c8e77de819cd99509eb8\": rpc error: code = NotFound desc = could not find container \"e6d10c7fa2a095839d70d6e147f0dcd7e2a67bee2257c8e77de819cd99509eb8\": container with ID starting with e6d10c7fa2a095839d70d6e147f0dcd7e2a67bee2257c8e77de819cd99509eb8 not found: ID does not exist"
Jul 20 20:05:54.439194 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:54.439173    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-n4zhv"]
Jul 20 20:05:54.444531 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:54.444514    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-n4zhv"]
Jul 20 20:05:55.175787 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:05:55.175759    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4939d813-2c85-4615-8425-bb61aa96f5cb" path="/var/lib/kubelet/pods/4939d813-2c85-4615-8425-bb61aa96f5cb/volumes"
Jul 20 20:06:17.275322 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.275285    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-snpr2"]
Jul 20 20:06:17.275736 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.275557    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4939d813-2c85-4615-8425-bb61aa96f5cb" containerName="authorino"
Jul 20 20:06:17.275736 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.275567    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4939d813-2c85-4615-8425-bb61aa96f5cb" containerName="authorino"
Jul 20 20:06:17.275736 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.275641    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4939d813-2c85-4615-8425-bb61aa96f5cb" containerName="authorino"
Jul 20 20:06:17.278150 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.278126    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-snpr2"
Jul 20 20:06:17.292066 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.292040    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-snpr2"]
Jul 20 20:06:17.345825 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.345781    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bs86\" (UniqueName: \"kubernetes.io/projected/55098b5e-bc72-4121-b168-8706920b3aee-kube-api-access-5bs86\") pod \"authorino-5d4b67c446-snpr2\" (UID: \"55098b5e-bc72-4121-b168-8706920b3aee\") " pod="kuadrant-system/authorino-5d4b67c446-snpr2"
Jul 20 20:06:17.446806 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.446767    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bs86\" (UniqueName: \"kubernetes.io/projected/55098b5e-bc72-4121-b168-8706920b3aee-kube-api-access-5bs86\") pod \"authorino-5d4b67c446-snpr2\" (UID: \"55098b5e-bc72-4121-b168-8706920b3aee\") " pod="kuadrant-system/authorino-5d4b67c446-snpr2"
Jul 20 20:06:17.456229 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.456200    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bs86\" (UniqueName: \"kubernetes.io/projected/55098b5e-bc72-4121-b168-8706920b3aee-kube-api-access-5bs86\") pod \"authorino-5d4b67c446-snpr2\" (UID: \"55098b5e-bc72-4121-b168-8706920b3aee\") " pod="kuadrant-system/authorino-5d4b67c446-snpr2"
Jul 20 20:06:17.503722 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.503688    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-snpr2"]
Jul 20 20:06:17.503915 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.503904    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-snpr2"
Jul 20 20:06:17.532609 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.532546    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6948d95bb-2d5s8"]
Jul 20 20:06:17.534642 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.534626    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6948d95bb-2d5s8"
Jul 20 20:06:17.543838 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.543812    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6948d95bb-2d5s8"]
Jul 20 20:06:17.633784 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.633754    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-snpr2"]
Jul 20 20:06:17.636390 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:06:17.636363    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55098b5e_bc72_4121_b168_8706920b3aee.slice/crio-548b1f6de17578633fd332b82b396db92a3e3a71b6de3b88050a7d74d836eeba WatchSource:0}: Error finding container 548b1f6de17578633fd332b82b396db92a3e3a71b6de3b88050a7d74d836eeba: Status 404 returned error can't find the container with id 548b1f6de17578633fd332b82b396db92a3e3a71b6de3b88050a7d74d836eeba
Jul 20 20:06:17.648736 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.648713    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25vs6\" (UniqueName: \"kubernetes.io/projected/8d488ec3-f976-4cbb-af49-17b8b73bfba7-kube-api-access-25vs6\") pod \"authorino-6948d95bb-2d5s8\" (UID: \"8d488ec3-f976-4cbb-af49-17b8b73bfba7\") " pod="kuadrant-system/authorino-6948d95bb-2d5s8"
Jul 20 20:06:17.749859 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.749823    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25vs6\" (UniqueName: \"kubernetes.io/projected/8d488ec3-f976-4cbb-af49-17b8b73bfba7-kube-api-access-25vs6\") pod \"authorino-6948d95bb-2d5s8\" (UID: \"8d488ec3-f976-4cbb-af49-17b8b73bfba7\") " pod="kuadrant-system/authorino-6948d95bb-2d5s8"
Jul 20 20:06:17.758574 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.758544    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25vs6\" (UniqueName: \"kubernetes.io/projected/8d488ec3-f976-4cbb-af49-17b8b73bfba7-kube-api-access-25vs6\") pod \"authorino-6948d95bb-2d5s8\" (UID: \"8d488ec3-f976-4cbb-af49-17b8b73bfba7\") " pod="kuadrant-system/authorino-6948d95bb-2d5s8"
Jul 20 20:06:17.806233 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.806200    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6948d95bb-2d5s8"]
Jul 20 20:06:17.806431 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.806420    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6948d95bb-2d5s8"
Jul 20 20:06:17.834450 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.834416    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6b6458566-ggjph"]
Jul 20 20:06:17.836480 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.836457    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b6458566-ggjph"
Jul 20 20:06:17.839063 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.839042    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 20 20:06:17.847811 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.847789    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b6458566-ggjph"]
Jul 20 20:06:17.940807 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.940770    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6948d95bb-2d5s8"]
Jul 20 20:06:17.943756 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:06:17.943722    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d488ec3_f976_4cbb_af49_17b8b73bfba7.slice/crio-cf3379bef323c3d209fd9bcfec9f091be119fc0cdbf2c18e6623a47c003e4d2f WatchSource:0}: Error finding container cf3379bef323c3d209fd9bcfec9f091be119fc0cdbf2c18e6623a47c003e4d2f: Status 404 returned error can't find the container with id cf3379bef323c3d209fd9bcfec9f091be119fc0cdbf2c18e6623a47c003e4d2f
Jul 20 20:06:17.951018 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.950993    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bc8aae13-55e2-4367-9c8a-55ceb5f80357-tls-cert\") pod \"authorino-6b6458566-ggjph\" (UID: \"bc8aae13-55e2-4367-9c8a-55ceb5f80357\") " pod="kuadrant-system/authorino-6b6458566-ggjph"
Jul 20 20:06:17.951155 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:17.951050    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqsrd\" (UniqueName: \"kubernetes.io/projected/bc8aae13-55e2-4367-9c8a-55ceb5f80357-kube-api-access-mqsrd\") pod \"authorino-6b6458566-ggjph\" (UID: \"bc8aae13-55e2-4367-9c8a-55ceb5f80357\") " pod="kuadrant-system/authorino-6b6458566-ggjph"
Jul 20 20:06:18.051941 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:18.051903    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bc8aae13-55e2-4367-9c8a-55ceb5f80357-tls-cert\") pod \"authorino-6b6458566-ggjph\" (UID: \"bc8aae13-55e2-4367-9c8a-55ceb5f80357\") " pod="kuadrant-system/authorino-6b6458566-ggjph"
Jul 20 20:06:18.052135 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:18.051991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqsrd\" (UniqueName: \"kubernetes.io/projected/bc8aae13-55e2-4367-9c8a-55ceb5f80357-kube-api-access-mqsrd\") pod \"authorino-6b6458566-ggjph\" (UID: \"bc8aae13-55e2-4367-9c8a-55ceb5f80357\") " pod="kuadrant-system/authorino-6b6458566-ggjph"
Jul 20 20:06:18.054313 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:18.054294    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bc8aae13-55e2-4367-9c8a-55ceb5f80357-tls-cert\") pod \"authorino-6b6458566-ggjph\" (UID: \"bc8aae13-55e2-4367-9c8a-55ceb5f80357\") " pod="kuadrant-system/authorino-6b6458566-ggjph"
Jul 20 20:06:18.060805 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:18.060760    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqsrd\" (UniqueName: \"kubernetes.io/projected/bc8aae13-55e2-4367-9c8a-55ceb5f80357-kube-api-access-mqsrd\") pod \"authorino-6b6458566-ggjph\" (UID: \"bc8aae13-55e2-4367-9c8a-55ceb5f80357\") " pod="kuadrant-system/authorino-6b6458566-ggjph"
Jul 20 20:06:18.146239 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:18.146205    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b6458566-ggjph"
Jul 20 20:06:18.277163 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:18.277133    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b6458566-ggjph"]
Jul 20 20:06:18.280413 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:06:18.280386    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc8aae13_55e2_4367_9c8a_55ceb5f80357.slice/crio-1929f517d708b271d8f610f0a7eb7686c8a55ec4f280ea4702a3f683d84915d8 WatchSource:0}: Error finding container 1929f517d708b271d8f610f0a7eb7686c8a55ec4f280ea4702a3f683d84915d8: Status 404 returned error can't find the container with id 1929f517d708b271d8f610f0a7eb7686c8a55ec4f280ea4702a3f683d84915d8
Jul 20 20:06:18.508593 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:18.508558    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6948d95bb-2d5s8" event={"ID":"8d488ec3-f976-4cbb-af49-17b8b73bfba7","Type":"ContainerStarted","Data":"1796072baf1e23cd7bcc921c895bb47c60871820da9a91ace96433e114190c15"}
Jul 20 20:06:18.508593 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:18.508599    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6948d95bb-2d5s8" event={"ID":"8d488ec3-f976-4cbb-af49-17b8b73bfba7","Type":"ContainerStarted","Data":"cf3379bef323c3d209fd9bcfec9f091be119fc0cdbf2c18e6623a47c003e4d2f"}
Jul 20 20:06:18.508844 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:18.508612    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6948d95bb-2d5s8" podUID="8d488ec3-f976-4cbb-af49-17b8b73bfba7" containerName="authorino" containerID="cri-o://1796072baf1e23cd7bcc921c895bb47c60871820da9a91ace96433e114190c15" gracePeriod=30
Jul 20 20:06:18.509880 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:18.509858    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b6458566-ggjph" event={"ID":"bc8aae13-55e2-4367-9c8a-55ceb5f80357","Type":"ContainerStarted","Data":"1929f517d708b271d8f610f0a7eb7686c8a55ec4f280ea4702a3f683d84915d8"}
Jul 20 20:06:18.511406 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:18.511369    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-snpr2" event={"ID":"55098b5e-bc72-4121-b168-8706920b3aee","Type":"ContainerStarted","Data":"f32cd07ce7df256328f906046f7df312fc4677bc33e6483225541e147ce2806c"}
Jul 20 20:06:18.511406 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:18.511406    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-snpr2" event={"ID":"55098b5e-bc72-4121-b168-8706920b3aee","Type":"ContainerStarted","Data":"548b1f6de17578633fd332b82b396db92a3e3a71b6de3b88050a7d74d836eeba"}
Jul 20 20:06:18.511613 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:18.511480    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-snpr2" podUID="55098b5e-bc72-4121-b168-8706920b3aee" containerName="authorino" containerID="cri-o://f32cd07ce7df256328f906046f7df312fc4677bc33e6483225541e147ce2806c" gracePeriod=30
Jul 20 20:06:18.525313 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:18.525269    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6948d95bb-2d5s8" podStartSLOduration=1.144776732 podStartE2EDuration="1.525253923s" totalImagesPullingTime="380.477191ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:06:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:06:18.524441941 +0000 UTC m=+593.788511988" watchObservedRunningTime="2026-07-20 20:06:18.525253923 +0000 UTC m=+593.789323946"
Jul 20 20:06:18.544799 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:18.544753    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-snpr2" podStartSLOduration=0.990134008 podStartE2EDuration="1.54473899s" totalImagesPullingTime="554.604982ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:06:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:06:18.543686301 +0000 UTC m=+593.807756348" watchObservedRunningTime="2026-07-20 20:06:18.54473899 +0000 UTC m=+593.808809037"
Jul 20 20:06:18.817766 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:18.817740    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6948d95bb-2d5s8"
Jul 20 20:06:18.841214 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:18.841118    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-snpr2"
Jul 20 20:06:18.857052 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:18.857017    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25vs6\" (UniqueName: \"kubernetes.io/projected/8d488ec3-f976-4cbb-af49-17b8b73bfba7-kube-api-access-25vs6\") pod \"8d488ec3-f976-4cbb-af49-17b8b73bfba7\" (UID: \"8d488ec3-f976-4cbb-af49-17b8b73bfba7\") "
Jul 20 20:06:18.859555 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:18.859526    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d488ec3-f976-4cbb-af49-17b8b73bfba7-kube-api-access-25vs6" (OuterVolumeSpecName: "kube-api-access-25vs6") pod "8d488ec3-f976-4cbb-af49-17b8b73bfba7" (UID: "8d488ec3-f976-4cbb-af49-17b8b73bfba7"). InnerVolumeSpecName "kube-api-access-25vs6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:06:18.957993 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:18.957940    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bs86\" (UniqueName: \"kubernetes.io/projected/55098b5e-bc72-4121-b168-8706920b3aee-kube-api-access-5bs86\") pod \"55098b5e-bc72-4121-b168-8706920b3aee\" (UID: \"55098b5e-bc72-4121-b168-8706920b3aee\") "
Jul 20 20:06:18.958168 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:18.958134    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25vs6\" (UniqueName: \"kubernetes.io/projected/8d488ec3-f976-4cbb-af49-17b8b73bfba7-kube-api-access-25vs6\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:06:18.959908 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:18.959886    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55098b5e-bc72-4121-b168-8706920b3aee-kube-api-access-5bs86" (OuterVolumeSpecName: "kube-api-access-5bs86") pod "55098b5e-bc72-4121-b168-8706920b3aee" (UID: "55098b5e-bc72-4121-b168-8706920b3aee"). InnerVolumeSpecName "kube-api-access-5bs86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:06:19.059347 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.059313    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bs86\" (UniqueName: \"kubernetes.io/projected/55098b5e-bc72-4121-b168-8706920b3aee-kube-api-access-5bs86\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:06:19.516982 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.516928    2580 generic.go:356] "Generic (PLEG): container finished" podID="55098b5e-bc72-4121-b168-8706920b3aee" containerID="f32cd07ce7df256328f906046f7df312fc4677bc33e6483225541e147ce2806c" exitCode=0
Jul 20 20:06:19.516982 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.516984    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-snpr2"
Jul 20 20:06:19.517504 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.516999    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-snpr2" event={"ID":"55098b5e-bc72-4121-b168-8706920b3aee","Type":"ContainerDied","Data":"f32cd07ce7df256328f906046f7df312fc4677bc33e6483225541e147ce2806c"}
Jul 20 20:06:19.517504 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.517045    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-snpr2" event={"ID":"55098b5e-bc72-4121-b168-8706920b3aee","Type":"ContainerDied","Data":"548b1f6de17578633fd332b82b396db92a3e3a71b6de3b88050a7d74d836eeba"}
Jul 20 20:06:19.517504 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.517069    2580 scope.go:117] "RemoveContainer" containerID="f32cd07ce7df256328f906046f7df312fc4677bc33e6483225541e147ce2806c"
Jul 20 20:06:19.518438 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.518326    2580 generic.go:356] "Generic (PLEG): container finished" podID="8d488ec3-f976-4cbb-af49-17b8b73bfba7" containerID="1796072baf1e23cd7bcc921c895bb47c60871820da9a91ace96433e114190c15" exitCode=0
Jul 20 20:06:19.518438 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.518358    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6948d95bb-2d5s8" event={"ID":"8d488ec3-f976-4cbb-af49-17b8b73bfba7","Type":"ContainerDied","Data":"1796072baf1e23cd7bcc921c895bb47c60871820da9a91ace96433e114190c15"}
Jul 20 20:06:19.518438 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.518380    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6948d95bb-2d5s8"
Jul 20 20:06:19.518438 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.518399    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6948d95bb-2d5s8" event={"ID":"8d488ec3-f976-4cbb-af49-17b8b73bfba7","Type":"ContainerDied","Data":"cf3379bef323c3d209fd9bcfec9f091be119fc0cdbf2c18e6623a47c003e4d2f"}
Jul 20 20:06:19.520079 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.519990    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b6458566-ggjph" event={"ID":"bc8aae13-55e2-4367-9c8a-55ceb5f80357","Type":"ContainerStarted","Data":"8ce156a6846042ca022b47a1e3547151fbb3ba2504e418dc15089206202c3010"}
Jul 20 20:06:19.525358 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.525345    2580 scope.go:117] "RemoveContainer" containerID="f32cd07ce7df256328f906046f7df312fc4677bc33e6483225541e147ce2806c"
Jul 20 20:06:19.525600 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:06:19.525584    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f32cd07ce7df256328f906046f7df312fc4677bc33e6483225541e147ce2806c\": container with ID starting with f32cd07ce7df256328f906046f7df312fc4677bc33e6483225541e147ce2806c not found: ID does not exist" containerID="f32cd07ce7df256328f906046f7df312fc4677bc33e6483225541e147ce2806c"
Jul 20 20:06:19.525641 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.525609    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f32cd07ce7df256328f906046f7df312fc4677bc33e6483225541e147ce2806c"} err="failed to get container status \"f32cd07ce7df256328f906046f7df312fc4677bc33e6483225541e147ce2806c\": rpc error: code = NotFound desc = could not find container \"f32cd07ce7df256328f906046f7df312fc4677bc33e6483225541e147ce2806c\": container with ID starting with f32cd07ce7df256328f906046f7df312fc4677bc33e6483225541e147ce2806c not found: ID does not exist"
Jul 20 20:06:19.525641 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.525626    2580 scope.go:117] "RemoveContainer" containerID="1796072baf1e23cd7bcc921c895bb47c60871820da9a91ace96433e114190c15"
Jul 20 20:06:19.532474 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.532454    2580 scope.go:117] "RemoveContainer" containerID="1796072baf1e23cd7bcc921c895bb47c60871820da9a91ace96433e114190c15"
Jul 20 20:06:19.532790 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:06:19.532763    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1796072baf1e23cd7bcc921c895bb47c60871820da9a91ace96433e114190c15\": container with ID starting with 1796072baf1e23cd7bcc921c895bb47c60871820da9a91ace96433e114190c15 not found: ID does not exist" containerID="1796072baf1e23cd7bcc921c895bb47c60871820da9a91ace96433e114190c15"
Jul 20 20:06:19.532946 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.532795    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1796072baf1e23cd7bcc921c895bb47c60871820da9a91ace96433e114190c15"} err="failed to get container status \"1796072baf1e23cd7bcc921c895bb47c60871820da9a91ace96433e114190c15\": rpc error: code = NotFound desc = could not find container \"1796072baf1e23cd7bcc921c895bb47c60871820da9a91ace96433e114190c15\": container with ID starting with 1796072baf1e23cd7bcc921c895bb47c60871820da9a91ace96433e114190c15 not found: ID does not exist"
Jul 20 20:06:19.535283 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.535260    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6948d95bb-2d5s8"]
Jul 20 20:06:19.539213 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.539195    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6948d95bb-2d5s8"]
Jul 20 20:06:19.550476 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.550458    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-snpr2"]
Jul 20 20:06:19.552719 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.552701    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-snpr2"]
Jul 20 20:06:19.572068 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.572023    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6b6458566-ggjph" podStartSLOduration=2.113095005 podStartE2EDuration="2.5720073s" totalImagesPullingTime="458.912295ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:06:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:06:19.570341795 +0000 UTC m=+594.834411840" watchObservedRunningTime="2026-07-20 20:06:19.5720073 +0000 UTC m=+594.836077714"
Jul 20 20:06:19.605999 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.605969    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7nxbt"]
Jul 20 20:06:19.606167 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.606148    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-7nxbt" podUID="9a41f580-2b76-45a6-a072-953c7c2e1317" containerName="authorino" containerID="cri-o://6bac81e4d29421d18a347d0a09f130ea3681029b8f43b47d6e708516cf66833e" gracePeriod=30
Jul 20 20:06:19.860689 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.860660    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7nxbt"
Jul 20 20:06:19.968414 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.968380    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxgzs\" (UniqueName: \"kubernetes.io/projected/9a41f580-2b76-45a6-a072-953c7c2e1317-kube-api-access-pxgzs\") pod \"9a41f580-2b76-45a6-a072-953c7c2e1317\" (UID: \"9a41f580-2b76-45a6-a072-953c7c2e1317\") "
Jul 20 20:06:19.970450 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:19.970423    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a41f580-2b76-45a6-a072-953c7c2e1317-kube-api-access-pxgzs" (OuterVolumeSpecName: "kube-api-access-pxgzs") pod "9a41f580-2b76-45a6-a072-953c7c2e1317" (UID: "9a41f580-2b76-45a6-a072-953c7c2e1317"). InnerVolumeSpecName "kube-api-access-pxgzs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:06:20.069260 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:20.069223    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxgzs\" (UniqueName: \"kubernetes.io/projected/9a41f580-2b76-45a6-a072-953c7c2e1317-kube-api-access-pxgzs\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:06:20.531149 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:20.531115    2580 generic.go:356] "Generic (PLEG): container finished" podID="9a41f580-2b76-45a6-a072-953c7c2e1317" containerID="6bac81e4d29421d18a347d0a09f130ea3681029b8f43b47d6e708516cf66833e" exitCode=0
Jul 20 20:06:20.531663 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:20.531145    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7nxbt" event={"ID":"9a41f580-2b76-45a6-a072-953c7c2e1317","Type":"ContainerDied","Data":"6bac81e4d29421d18a347d0a09f130ea3681029b8f43b47d6e708516cf66833e"}
Jul 20 20:06:20.531663 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:20.531183    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7nxbt"
Jul 20 20:06:20.531663 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:20.531203    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7nxbt" event={"ID":"9a41f580-2b76-45a6-a072-953c7c2e1317","Type":"ContainerDied","Data":"b5e31fde5ef3809b499b27de2c4f9eb1141e7d6ac7b4593fca1d2e812c4ea3a3"}
Jul 20 20:06:20.531663 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:20.531225    2580 scope.go:117] "RemoveContainer" containerID="6bac81e4d29421d18a347d0a09f130ea3681029b8f43b47d6e708516cf66833e"
Jul 20 20:06:20.539605 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:20.539589    2580 scope.go:117] "RemoveContainer" containerID="6bac81e4d29421d18a347d0a09f130ea3681029b8f43b47d6e708516cf66833e"
Jul 20 20:06:20.539842 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:06:20.539826    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bac81e4d29421d18a347d0a09f130ea3681029b8f43b47d6e708516cf66833e\": container with ID starting with 6bac81e4d29421d18a347d0a09f130ea3681029b8f43b47d6e708516cf66833e not found: ID does not exist" containerID="6bac81e4d29421d18a347d0a09f130ea3681029b8f43b47d6e708516cf66833e"
Jul 20 20:06:20.539884 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:20.539849    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bac81e4d29421d18a347d0a09f130ea3681029b8f43b47d6e708516cf66833e"} err="failed to get container status \"6bac81e4d29421d18a347d0a09f130ea3681029b8f43b47d6e708516cf66833e\": rpc error: code = NotFound desc = could not find container \"6bac81e4d29421d18a347d0a09f130ea3681029b8f43b47d6e708516cf66833e\": container with ID starting with 6bac81e4d29421d18a347d0a09f130ea3681029b8f43b47d6e708516cf66833e not found: ID does not exist"
Jul 20 20:06:20.552558 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:20.552535    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7nxbt"]
Jul 20 20:06:20.556885 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:20.556861    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7nxbt"]
Jul 20 20:06:21.174976 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:21.174938    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55098b5e-bc72-4121-b168-8706920b3aee" path="/var/lib/kubelet/pods/55098b5e-bc72-4121-b168-8706920b3aee/volumes"
Jul 20 20:06:21.175284 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:21.175271    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d488ec3-f976-4cbb-af49-17b8b73bfba7" path="/var/lib/kubelet/pods/8d488ec3-f976-4cbb-af49-17b8b73bfba7/volumes"
Jul 20 20:06:21.175539 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:21.175528    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a41f580-2b76-45a6-a072-953c7c2e1317" path="/var/lib/kubelet/pods/9a41f580-2b76-45a6-a072-953c7c2e1317/volumes"
Jul 20 20:06:25.065843 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:25.065816    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 20:06:25.068997 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:06:25.068976    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 20:07:34.231408 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.231378    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"]
Jul 20 20:07:34.231813 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.231654    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55098b5e-bc72-4121-b168-8706920b3aee" containerName="authorino"
Jul 20 20:07:34.231813 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.231665    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="55098b5e-bc72-4121-b168-8706920b3aee" containerName="authorino"
Jul 20 20:07:34.231813 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.231676    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a41f580-2b76-45a6-a072-953c7c2e1317" containerName="authorino"
Jul 20 20:07:34.231813 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.231682    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a41f580-2b76-45a6-a072-953c7c2e1317" containerName="authorino"
Jul 20 20:07:34.231813 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.231691    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d488ec3-f976-4cbb-af49-17b8b73bfba7" containerName="authorino"
Jul 20 20:07:34.231813 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.231696    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d488ec3-f976-4cbb-af49-17b8b73bfba7" containerName="authorino"
Jul 20 20:07:34.231813 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.231751    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a41f580-2b76-45a6-a072-953c7c2e1317" containerName="authorino"
Jul 20 20:07:34.231813 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.231760    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="55098b5e-bc72-4121-b168-8706920b3aee" containerName="authorino"
Jul 20 20:07:34.231813 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.231769    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d488ec3-f976-4cbb-af49-17b8b73bfba7" containerName="authorino"
Jul 20 20:07:34.234598 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.234582    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:34.237657 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.237632    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 20 20:07:34.237787 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.237701    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 20 20:07:34.237787 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.237774    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-z9f7c\""
Jul 20 20:07:34.237903 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.237846    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 20 20:07:34.246977 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.246942    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"]
Jul 20 20:07:34.361051 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.361020    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd2b0998-57e0-42a1-9bf5-ec2808f01eef-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jjmtz\" (UID: \"bd2b0998-57e0-42a1-9bf5-ec2808f01eef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:34.361051 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.361053    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd2b0998-57e0-42a1-9bf5-ec2808f01eef-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jjmtz\" (UID: \"bd2b0998-57e0-42a1-9bf5-ec2808f01eef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:34.361262 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.361080    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd2b0998-57e0-42a1-9bf5-ec2808f01eef-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jjmtz\" (UID: \"bd2b0998-57e0-42a1-9bf5-ec2808f01eef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:34.361262 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.361132    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkn29\" (UniqueName: \"kubernetes.io/projected/bd2b0998-57e0-42a1-9bf5-ec2808f01eef-kube-api-access-jkn29\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jjmtz\" (UID: \"bd2b0998-57e0-42a1-9bf5-ec2808f01eef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:34.361262 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.361217    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd2b0998-57e0-42a1-9bf5-ec2808f01eef-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jjmtz\" (UID: \"bd2b0998-57e0-42a1-9bf5-ec2808f01eef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:34.361354 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.361267    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd2b0998-57e0-42a1-9bf5-ec2808f01eef-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jjmtz\" (UID: \"bd2b0998-57e0-42a1-9bf5-ec2808f01eef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:34.461829 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.461792    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd2b0998-57e0-42a1-9bf5-ec2808f01eef-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jjmtz\" (UID: \"bd2b0998-57e0-42a1-9bf5-ec2808f01eef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:34.461829 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.461835    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd2b0998-57e0-42a1-9bf5-ec2808f01eef-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jjmtz\" (UID: \"bd2b0998-57e0-42a1-9bf5-ec2808f01eef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:34.462113 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.461852    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd2b0998-57e0-42a1-9bf5-ec2808f01eef-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jjmtz\" (UID: \"bd2b0998-57e0-42a1-9bf5-ec2808f01eef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:34.462113 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.461874    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd2b0998-57e0-42a1-9bf5-ec2808f01eef-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jjmtz\" (UID: \"bd2b0998-57e0-42a1-9bf5-ec2808f01eef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:34.462113 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.461891    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkn29\" (UniqueName: \"kubernetes.io/projected/bd2b0998-57e0-42a1-9bf5-ec2808f01eef-kube-api-access-jkn29\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jjmtz\" (UID: \"bd2b0998-57e0-42a1-9bf5-ec2808f01eef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:34.462113 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.461925    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd2b0998-57e0-42a1-9bf5-ec2808f01eef-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jjmtz\" (UID: \"bd2b0998-57e0-42a1-9bf5-ec2808f01eef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:34.462300 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.462279    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd2b0998-57e0-42a1-9bf5-ec2808f01eef-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jjmtz\" (UID: \"bd2b0998-57e0-42a1-9bf5-ec2808f01eef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:34.462357 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.462319    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd2b0998-57e0-42a1-9bf5-ec2808f01eef-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jjmtz\" (UID: \"bd2b0998-57e0-42a1-9bf5-ec2808f01eef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:34.462407 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.462361    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd2b0998-57e0-42a1-9bf5-ec2808f01eef-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jjmtz\" (UID: \"bd2b0998-57e0-42a1-9bf5-ec2808f01eef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:34.464222 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.464204    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd2b0998-57e0-42a1-9bf5-ec2808f01eef-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jjmtz\" (UID: \"bd2b0998-57e0-42a1-9bf5-ec2808f01eef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:34.464334 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.464316    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd2b0998-57e0-42a1-9bf5-ec2808f01eef-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jjmtz\" (UID: \"bd2b0998-57e0-42a1-9bf5-ec2808f01eef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:34.472071 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.472043    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkn29\" (UniqueName: \"kubernetes.io/projected/bd2b0998-57e0-42a1-9bf5-ec2808f01eef-kube-api-access-jkn29\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jjmtz\" (UID: \"bd2b0998-57e0-42a1-9bf5-ec2808f01eef\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:34.545262 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.545161    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:34.674761 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.674735    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"]
Jul 20 20:07:34.676584 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:07:34.676542    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd2b0998_57e0_42a1_9bf5_ec2808f01eef.slice/crio-f678116a191830b098a1cf442fa3840dbe269b377b8e0f56c3e251582586e6c4 WatchSource:0}: Error finding container f678116a191830b098a1cf442fa3840dbe269b377b8e0f56c3e251582586e6c4: Status 404 returned error can't find the container with id f678116a191830b098a1cf442fa3840dbe269b377b8e0f56c3e251582586e6c4
Jul 20 20:07:34.677869 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.677853    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 20:07:34.805782 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:34.805748    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz" event={"ID":"bd2b0998-57e0-42a1-9bf5-ec2808f01eef","Type":"ContainerStarted","Data":"f678116a191830b098a1cf442fa3840dbe269b377b8e0f56c3e251582586e6c4"}
Jul 20 20:07:36.815444 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:36.815400    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz" event={"ID":"bd2b0998-57e0-42a1-9bf5-ec2808f01eef","Type":"ContainerStarted","Data":"e4d1fa15dcb09743ec7955de797a1ce8b7a25d476606b59f22a4c176fa31b0a9"}
Jul 20 20:07:36.844488 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:36.844431    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz" podStartSLOduration=1.725554962 podStartE2EDuration="2.84441384s" totalImagesPullingTime="1.118858878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:07:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:07:36.843122614 +0000 UTC m=+672.107192662" watchObservedRunningTime="2026-07-20 20:07:36.84441384 +0000 UTC m=+672.108483885"
Jul 20 20:07:44.545314 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:44.545280    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:44.545724 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:44.545343    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:44.557845 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:44.557815    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:07:44.859071 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:07:44.859044    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jjmtz"
Jul 20 20:08:10.642130 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.642099    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"]
Jul 20 20:08:10.646820 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.646804    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:10.649988 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.649969    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 20 20:08:10.670421 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.670387    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"]
Jul 20 20:08:10.675262 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.675238    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a555a3f0-e824-4a93-9946-8ffbdf4e76b7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-679wt\" (UID: \"a555a3f0-e824-4a93-9946-8ffbdf4e76b7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:10.675365 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.675271    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h64j4\" (UniqueName: \"kubernetes.io/projected/a555a3f0-e824-4a93-9946-8ffbdf4e76b7-kube-api-access-h64j4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-679wt\" (UID: \"a555a3f0-e824-4a93-9946-8ffbdf4e76b7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:10.675365 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.675320    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a555a3f0-e824-4a93-9946-8ffbdf4e76b7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-679wt\" (UID: \"a555a3f0-e824-4a93-9946-8ffbdf4e76b7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:10.675365 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.675341    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a555a3f0-e824-4a93-9946-8ffbdf4e76b7-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-679wt\" (UID: \"a555a3f0-e824-4a93-9946-8ffbdf4e76b7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:10.675365 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.675355    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a555a3f0-e824-4a93-9946-8ffbdf4e76b7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-679wt\" (UID: \"a555a3f0-e824-4a93-9946-8ffbdf4e76b7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:10.675490 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.675411    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a555a3f0-e824-4a93-9946-8ffbdf4e76b7-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-679wt\" (UID: \"a555a3f0-e824-4a93-9946-8ffbdf4e76b7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:10.775856 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.775821    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a555a3f0-e824-4a93-9946-8ffbdf4e76b7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-679wt\" (UID: \"a555a3f0-e824-4a93-9946-8ffbdf4e76b7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:10.775856 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.775860    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a555a3f0-e824-4a93-9946-8ffbdf4e76b7-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-679wt\" (UID: \"a555a3f0-e824-4a93-9946-8ffbdf4e76b7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:10.776097 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.775877    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a555a3f0-e824-4a93-9946-8ffbdf4e76b7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-679wt\" (UID: \"a555a3f0-e824-4a93-9946-8ffbdf4e76b7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:10.776097 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.776009    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a555a3f0-e824-4a93-9946-8ffbdf4e76b7-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-679wt\" (UID: \"a555a3f0-e824-4a93-9946-8ffbdf4e76b7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:10.776097 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.776087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a555a3f0-e824-4a93-9946-8ffbdf4e76b7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-679wt\" (UID: \"a555a3f0-e824-4a93-9946-8ffbdf4e76b7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:10.776235 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.776121    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h64j4\" (UniqueName: \"kubernetes.io/projected/a555a3f0-e824-4a93-9946-8ffbdf4e76b7-kube-api-access-h64j4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-679wt\" (UID: \"a555a3f0-e824-4a93-9946-8ffbdf4e76b7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:10.776290 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.776276    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a555a3f0-e824-4a93-9946-8ffbdf4e76b7-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-679wt\" (UID: \"a555a3f0-e824-4a93-9946-8ffbdf4e76b7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:10.776341 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.776320    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a555a3f0-e824-4a93-9946-8ffbdf4e76b7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-679wt\" (UID: \"a555a3f0-e824-4a93-9946-8ffbdf4e76b7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:10.776450 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.776434    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a555a3f0-e824-4a93-9946-8ffbdf4e76b7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-679wt\" (UID: \"a555a3f0-e824-4a93-9946-8ffbdf4e76b7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:10.778229 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.778204    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a555a3f0-e824-4a93-9946-8ffbdf4e76b7-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-679wt\" (UID: \"a555a3f0-e824-4a93-9946-8ffbdf4e76b7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:10.778471 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.778456    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a555a3f0-e824-4a93-9946-8ffbdf4e76b7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-679wt\" (UID: \"a555a3f0-e824-4a93-9946-8ffbdf4e76b7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:10.785799 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.785777    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h64j4\" (UniqueName: \"kubernetes.io/projected/a555a3f0-e824-4a93-9946-8ffbdf4e76b7-kube-api-access-h64j4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-679wt\" (UID: \"a555a3f0-e824-4a93-9946-8ffbdf4e76b7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:10.955560 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:10.955475    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:11.086806 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:11.086783    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"]
Jul 20 20:08:11.089154 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:08:11.089128    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda555a3f0_e824_4a93_9946_8ffbdf4e76b7.slice/crio-b0467fe17d02a8480f6b5331de32483fd0e06ca686f745a75e856b1628d7fbbd WatchSource:0}: Error finding container b0467fe17d02a8480f6b5331de32483fd0e06ca686f745a75e856b1628d7fbbd: Status 404 returned error can't find the container with id b0467fe17d02a8480f6b5331de32483fd0e06ca686f745a75e856b1628d7fbbd
Jul 20 20:08:11.953691 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:11.953655    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt" event={"ID":"a555a3f0-e824-4a93-9946-8ffbdf4e76b7","Type":"ContainerStarted","Data":"4a2484c0b06074f9b23a1bd4ca00ea3304d564268af42bce8d59039ef4b62750"}
Jul 20 20:08:11.953691 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:11.953695    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt" event={"ID":"a555a3f0-e824-4a93-9946-8ffbdf4e76b7","Type":"ContainerStarted","Data":"b0467fe17d02a8480f6b5331de32483fd0e06ca686f745a75e856b1628d7fbbd"}
Jul 20 20:08:11.990064 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:11.990012    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt" podStartSLOduration=1.768548861 podStartE2EDuration="1.989997194s" totalImagesPullingTime="221.448333ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:08:11.988481136 +0000 UTC m=+707.252551184" watchObservedRunningTime="2026-07-20 20:08:11.989997194 +0000 UTC m=+707.254067239"
Jul 20 20:08:20.955754 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:20.955667    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:20.955754 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:20.955719    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:20.968395 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:20.968372    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:08:20.999680 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:08:20.999653    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-679wt"
Jul 20 20:09:38.262550 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:38.262516    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd897b9f-2dwkq"]
Jul 20 20:09:38.265578 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:38.265561    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd897b9f-2dwkq"
Jul 20 20:09:38.274212 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:38.274185    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd897b9f-2dwkq"]
Jul 20 20:09:38.406738 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:38.406708    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx6jv\" (UniqueName: \"kubernetes.io/projected/e4bf2449-c483-4a6b-9f03-1f6949ef6056-kube-api-access-dx6jv\") pod \"authorino-58cd897b9f-2dwkq\" (UID: \"e4bf2449-c483-4a6b-9f03-1f6949ef6056\") " pod="kuadrant-system/authorino-58cd897b9f-2dwkq"
Jul 20 20:09:38.406908 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:38.406743    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e4bf2449-c483-4a6b-9f03-1f6949ef6056-tls-cert\") pod \"authorino-58cd897b9f-2dwkq\" (UID: \"e4bf2449-c483-4a6b-9f03-1f6949ef6056\") " pod="kuadrant-system/authorino-58cd897b9f-2dwkq"
Jul 20 20:09:38.507921 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:38.507882    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx6jv\" (UniqueName: \"kubernetes.io/projected/e4bf2449-c483-4a6b-9f03-1f6949ef6056-kube-api-access-dx6jv\") pod \"authorino-58cd897b9f-2dwkq\" (UID: \"e4bf2449-c483-4a6b-9f03-1f6949ef6056\") " pod="kuadrant-system/authorino-58cd897b9f-2dwkq"
Jul 20 20:09:38.507921 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:38.507924    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e4bf2449-c483-4a6b-9f03-1f6949ef6056-tls-cert\") pod \"authorino-58cd897b9f-2dwkq\" (UID: \"e4bf2449-c483-4a6b-9f03-1f6949ef6056\") " pod="kuadrant-system/authorino-58cd897b9f-2dwkq"
Jul 20 20:09:38.510279 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:38.510260    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e4bf2449-c483-4a6b-9f03-1f6949ef6056-tls-cert\") pod \"authorino-58cd897b9f-2dwkq\" (UID: \"e4bf2449-c483-4a6b-9f03-1f6949ef6056\") " pod="kuadrant-system/authorino-58cd897b9f-2dwkq"
Jul 20 20:09:38.517574 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:38.517521    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx6jv\" (UniqueName: \"kubernetes.io/projected/e4bf2449-c483-4a6b-9f03-1f6949ef6056-kube-api-access-dx6jv\") pod \"authorino-58cd897b9f-2dwkq\" (UID: \"e4bf2449-c483-4a6b-9f03-1f6949ef6056\") " pod="kuadrant-system/authorino-58cd897b9f-2dwkq"
Jul 20 20:09:38.574268 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:38.574232    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd897b9f-2dwkq"
Jul 20 20:09:38.712762 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:38.712729    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd897b9f-2dwkq"]
Jul 20 20:09:38.716728 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:09:38.716696    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4bf2449_c483_4a6b_9f03_1f6949ef6056.slice/crio-cb708f7eff37be1e6124c7ebb6570568a3634b5672e346d2f0000319f1cca740 WatchSource:0}: Error finding container cb708f7eff37be1e6124c7ebb6570568a3634b5672e346d2f0000319f1cca740: Status 404 returned error can't find the container with id cb708f7eff37be1e6124c7ebb6570568a3634b5672e346d2f0000319f1cca740
Jul 20 20:09:39.301597 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:39.301563    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd897b9f-2dwkq" event={"ID":"e4bf2449-c483-4a6b-9f03-1f6949ef6056","Type":"ContainerStarted","Data":"9ac86d3c6a6d089157c7c710217fad3b88353ceee4d5594a69b004b58b65ac0c"}
Jul 20 20:09:39.302063 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:39.301608    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd897b9f-2dwkq" event={"ID":"e4bf2449-c483-4a6b-9f03-1f6949ef6056","Type":"ContainerStarted","Data":"cb708f7eff37be1e6124c7ebb6570568a3634b5672e346d2f0000319f1cca740"}
Jul 20 20:09:39.320928 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:39.320876    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd897b9f-2dwkq" podStartSLOduration=0.846528234 podStartE2EDuration="1.320858854s" totalImagesPullingTime="474.33062ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:09:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:09:39.319162034 +0000 UTC m=+794.583232081" watchObservedRunningTime="2026-07-20 20:09:39.320858854 +0000 UTC m=+794.584928900"
Jul 20 20:09:39.350913 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:39.350860    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6b6458566-ggjph"]
Jul 20 20:09:39.351128 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:39.351095    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6b6458566-ggjph" podUID="bc8aae13-55e2-4367-9c8a-55ceb5f80357" containerName="authorino" containerID="cri-o://8ce156a6846042ca022b47a1e3547151fbb3ba2504e418dc15089206202c3010" gracePeriod=30
Jul 20 20:09:39.605745 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:39.605717    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b6458566-ggjph"
Jul 20 20:09:39.718981 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:39.718879    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqsrd\" (UniqueName: \"kubernetes.io/projected/bc8aae13-55e2-4367-9c8a-55ceb5f80357-kube-api-access-mqsrd\") pod \"bc8aae13-55e2-4367-9c8a-55ceb5f80357\" (UID: \"bc8aae13-55e2-4367-9c8a-55ceb5f80357\") "
Jul 20 20:09:39.719150 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:39.719003    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bc8aae13-55e2-4367-9c8a-55ceb5f80357-tls-cert\") pod \"bc8aae13-55e2-4367-9c8a-55ceb5f80357\" (UID: \"bc8aae13-55e2-4367-9c8a-55ceb5f80357\") "
Jul 20 20:09:39.720953 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:39.720922    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc8aae13-55e2-4367-9c8a-55ceb5f80357-kube-api-access-mqsrd" (OuterVolumeSpecName: "kube-api-access-mqsrd") pod "bc8aae13-55e2-4367-9c8a-55ceb5f80357" (UID: "bc8aae13-55e2-4367-9c8a-55ceb5f80357"). InnerVolumeSpecName "kube-api-access-mqsrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:09:39.728383 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:39.728361    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc8aae13-55e2-4367-9c8a-55ceb5f80357-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "bc8aae13-55e2-4367-9c8a-55ceb5f80357" (UID: "bc8aae13-55e2-4367-9c8a-55ceb5f80357"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 20:09:39.819877 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:39.819841    2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bc8aae13-55e2-4367-9c8a-55ceb5f80357-tls-cert\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:09:39.819877 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:39.819872    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqsrd\" (UniqueName: \"kubernetes.io/projected/bc8aae13-55e2-4367-9c8a-55ceb5f80357-kube-api-access-mqsrd\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:09:40.307740 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:40.307703    2580 generic.go:356] "Generic (PLEG): container finished" podID="bc8aae13-55e2-4367-9c8a-55ceb5f80357" containerID="8ce156a6846042ca022b47a1e3547151fbb3ba2504e418dc15089206202c3010" exitCode=0
Jul 20 20:09:40.308208 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:40.307759    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b6458566-ggjph"
Jul 20 20:09:40.308208 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:40.307792    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b6458566-ggjph" event={"ID":"bc8aae13-55e2-4367-9c8a-55ceb5f80357","Type":"ContainerDied","Data":"8ce156a6846042ca022b47a1e3547151fbb3ba2504e418dc15089206202c3010"}
Jul 20 20:09:40.308208 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:40.307835    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b6458566-ggjph" event={"ID":"bc8aae13-55e2-4367-9c8a-55ceb5f80357","Type":"ContainerDied","Data":"1929f517d708b271d8f610f0a7eb7686c8a55ec4f280ea4702a3f683d84915d8"}
Jul 20 20:09:40.308208 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:40.307852    2580 scope.go:117] "RemoveContainer" containerID="8ce156a6846042ca022b47a1e3547151fbb3ba2504e418dc15089206202c3010"
Jul 20 20:09:40.319952 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:40.319931    2580 scope.go:117] "RemoveContainer" containerID="8ce156a6846042ca022b47a1e3547151fbb3ba2504e418dc15089206202c3010"
Jul 20 20:09:40.320243 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:09:40.320226    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ce156a6846042ca022b47a1e3547151fbb3ba2504e418dc15089206202c3010\": container with ID starting with 8ce156a6846042ca022b47a1e3547151fbb3ba2504e418dc15089206202c3010 not found: ID does not exist" containerID="8ce156a6846042ca022b47a1e3547151fbb3ba2504e418dc15089206202c3010"
Jul 20 20:09:40.320298 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:40.320253    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ce156a6846042ca022b47a1e3547151fbb3ba2504e418dc15089206202c3010"} err="failed to get container status \"8ce156a6846042ca022b47a1e3547151fbb3ba2504e418dc15089206202c3010\": rpc error: code = NotFound desc = could not find container \"8ce156a6846042ca022b47a1e3547151fbb3ba2504e418dc15089206202c3010\": container with ID starting with 8ce156a6846042ca022b47a1e3547151fbb3ba2504e418dc15089206202c3010 not found: ID does not exist"
Jul 20 20:09:40.335902 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:40.335875    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6b6458566-ggjph"]
Jul 20 20:09:40.337065 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:40.337044    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6b6458566-ggjph"]
Jul 20 20:09:41.176610 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:09:41.176568    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc8aae13-55e2-4367-9c8a-55ceb5f80357" path="/var/lib/kubelet/pods/bc8aae13-55e2-4367-9c8a-55ceb5f80357/volumes"
Jul 20 20:11:25.088387 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:25.088305    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 20:11:25.090272 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:25.090251    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 20:11:28.182642 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:28.182607    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77bdf94859-pxtr7"]
Jul 20 20:11:28.183016 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:28.182895    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc8aae13-55e2-4367-9c8a-55ceb5f80357" containerName="authorino"
Jul 20 20:11:28.183016 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:28.182907    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc8aae13-55e2-4367-9c8a-55ceb5f80357" containerName="authorino"
Jul 20 20:11:28.183016 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:28.182988    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc8aae13-55e2-4367-9c8a-55ceb5f80357" containerName="authorino"
Jul 20 20:11:28.185662 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:28.185647    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77bdf94859-pxtr7"
Jul 20 20:11:28.189883 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:28.189863    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 20 20:11:28.190111 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:28.190094    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4qdn7\""
Jul 20 20:11:28.191287 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:28.191271    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 20 20:11:28.205307 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:28.205286    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77bdf94859-pxtr7"]
Jul 20 20:11:28.288327 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:28.288288    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftp55\" (UniqueName: \"kubernetes.io/projected/02d894ae-8ac2-47e8-a27e-d075bf99ea3d-kube-api-access-ftp55\") pod \"maas-controller-77bdf94859-pxtr7\" (UID: \"02d894ae-8ac2-47e8-a27e-d075bf99ea3d\") " pod="opendatahub/maas-controller-77bdf94859-pxtr7"
Jul 20 20:11:28.288531 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:28.288386    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/02d894ae-8ac2-47e8-a27e-d075bf99ea3d-webhook-cert\") pod \"maas-controller-77bdf94859-pxtr7\" (UID: \"02d894ae-8ac2-47e8-a27e-d075bf99ea3d\") " pod="opendatahub/maas-controller-77bdf94859-pxtr7"
Jul 20 20:11:28.389610 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:28.389573    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftp55\" (UniqueName: \"kubernetes.io/projected/02d894ae-8ac2-47e8-a27e-d075bf99ea3d-kube-api-access-ftp55\") pod \"maas-controller-77bdf94859-pxtr7\" (UID: \"02d894ae-8ac2-47e8-a27e-d075bf99ea3d\") " pod="opendatahub/maas-controller-77bdf94859-pxtr7"
Jul 20 20:11:28.389790 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:28.389638    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/02d894ae-8ac2-47e8-a27e-d075bf99ea3d-webhook-cert\") pod \"maas-controller-77bdf94859-pxtr7\" (UID: \"02d894ae-8ac2-47e8-a27e-d075bf99ea3d\") " pod="opendatahub/maas-controller-77bdf94859-pxtr7"
Jul 20 20:11:28.392090 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:28.392067    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/02d894ae-8ac2-47e8-a27e-d075bf99ea3d-webhook-cert\") pod \"maas-controller-77bdf94859-pxtr7\" (UID: \"02d894ae-8ac2-47e8-a27e-d075bf99ea3d\") " pod="opendatahub/maas-controller-77bdf94859-pxtr7"
Jul 20 20:11:28.399264 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:28.399245    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftp55\" (UniqueName: \"kubernetes.io/projected/02d894ae-8ac2-47e8-a27e-d075bf99ea3d-kube-api-access-ftp55\") pod \"maas-controller-77bdf94859-pxtr7\" (UID: \"02d894ae-8ac2-47e8-a27e-d075bf99ea3d\") " pod="opendatahub/maas-controller-77bdf94859-pxtr7"
Jul 20 20:11:28.494905 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:28.494805    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77bdf94859-pxtr7"
Jul 20 20:11:28.627320 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:28.627294    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77bdf94859-pxtr7"]
Jul 20 20:11:28.629881 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:11:28.629853    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02d894ae_8ac2_47e8_a27e_d075bf99ea3d.slice/crio-60d8bef74aabe67651c8eb9d41f36592c191b6e18e3018387d46c6586f493ec2 WatchSource:0}: Error finding container 60d8bef74aabe67651c8eb9d41f36592c191b6e18e3018387d46c6586f493ec2: Status 404 returned error can't find the container with id 60d8bef74aabe67651c8eb9d41f36592c191b6e18e3018387d46c6586f493ec2
Jul 20 20:11:28.745710 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:28.745621    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77bdf94859-pxtr7" event={"ID":"02d894ae-8ac2-47e8-a27e-d075bf99ea3d","Type":"ContainerStarted","Data":"60d8bef74aabe67651c8eb9d41f36592c191b6e18e3018387d46c6586f493ec2"}
Jul 20 20:11:31.762362 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:31.762323    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77bdf94859-pxtr7" event={"ID":"02d894ae-8ac2-47e8-a27e-d075bf99ea3d","Type":"ContainerStarted","Data":"ac2f07ad2bbfd556d746510aa35f20828c69c84022e907d3dfb7d76b2ba54256"}
Jul 20 20:11:31.762762 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:31.762447    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77bdf94859-pxtr7"
Jul 20 20:11:31.782548 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:31.782497    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77bdf94859-pxtr7" podStartSLOduration=1.5227830610000002 podStartE2EDuration="3.782482257s" totalImagesPullingTime="2.259699196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:11:31.780729138 +0000 UTC m=+907.044799183" watchObservedRunningTime="2026-07-20 20:11:31.782482257 +0000 UTC m=+907.046552302"
Jul 20 20:11:42.768780 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:42.768748    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77bdf94859-pxtr7"
Jul 20 20:11:47.438645 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:47.438613    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77bdf94859-pxtr7"]
Jul 20 20:11:47.439087 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:47.438803    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-77bdf94859-pxtr7" podUID="02d894ae-8ac2-47e8-a27e-d075bf99ea3d" containerName="manager" containerID="cri-o://ac2f07ad2bbfd556d746510aa35f20828c69c84022e907d3dfb7d76b2ba54256" gracePeriod=30
Jul 20 20:11:47.676476 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:47.676448    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77bdf94859-pxtr7"
Jul 20 20:11:47.753257 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:47.753173    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftp55\" (UniqueName: \"kubernetes.io/projected/02d894ae-8ac2-47e8-a27e-d075bf99ea3d-kube-api-access-ftp55\") pod \"02d894ae-8ac2-47e8-a27e-d075bf99ea3d\" (UID: \"02d894ae-8ac2-47e8-a27e-d075bf99ea3d\") "
Jul 20 20:11:47.753421 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:47.753280    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/02d894ae-8ac2-47e8-a27e-d075bf99ea3d-webhook-cert\") pod \"02d894ae-8ac2-47e8-a27e-d075bf99ea3d\" (UID: \"02d894ae-8ac2-47e8-a27e-d075bf99ea3d\") "
Jul 20 20:11:47.755568 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:47.755533    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02d894ae-8ac2-47e8-a27e-d075bf99ea3d-kube-api-access-ftp55" (OuterVolumeSpecName: "kube-api-access-ftp55") pod "02d894ae-8ac2-47e8-a27e-d075bf99ea3d" (UID: "02d894ae-8ac2-47e8-a27e-d075bf99ea3d"). InnerVolumeSpecName "kube-api-access-ftp55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:11:47.755673 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:47.755568    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02d894ae-8ac2-47e8-a27e-d075bf99ea3d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "02d894ae-8ac2-47e8-a27e-d075bf99ea3d" (UID: "02d894ae-8ac2-47e8-a27e-d075bf99ea3d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 20:11:47.829365 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:47.829333    2580 generic.go:356] "Generic (PLEG): container finished" podID="02d894ae-8ac2-47e8-a27e-d075bf99ea3d" containerID="ac2f07ad2bbfd556d746510aa35f20828c69c84022e907d3dfb7d76b2ba54256" exitCode=0
Jul 20 20:11:47.829525 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:47.829378    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77bdf94859-pxtr7"
Jul 20 20:11:47.829525 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:47.829413    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77bdf94859-pxtr7" event={"ID":"02d894ae-8ac2-47e8-a27e-d075bf99ea3d","Type":"ContainerDied","Data":"ac2f07ad2bbfd556d746510aa35f20828c69c84022e907d3dfb7d76b2ba54256"}
Jul 20 20:11:47.829525 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:47.829467    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77bdf94859-pxtr7" event={"ID":"02d894ae-8ac2-47e8-a27e-d075bf99ea3d","Type":"ContainerDied","Data":"60d8bef74aabe67651c8eb9d41f36592c191b6e18e3018387d46c6586f493ec2"}
Jul 20 20:11:47.829525 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:47.829489    2580 scope.go:117] "RemoveContainer" containerID="ac2f07ad2bbfd556d746510aa35f20828c69c84022e907d3dfb7d76b2ba54256"
Jul 20 20:11:47.838182 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:47.838132    2580 scope.go:117] "RemoveContainer" containerID="ac2f07ad2bbfd556d746510aa35f20828c69c84022e907d3dfb7d76b2ba54256"
Jul 20 20:11:47.838395 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:11:47.838378    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac2f07ad2bbfd556d746510aa35f20828c69c84022e907d3dfb7d76b2ba54256\": container with ID starting with ac2f07ad2bbfd556d746510aa35f20828c69c84022e907d3dfb7d76b2ba54256 not found: ID does not exist" containerID="ac2f07ad2bbfd556d746510aa35f20828c69c84022e907d3dfb7d76b2ba54256"
Jul 20 20:11:47.838442 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:47.838404    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac2f07ad2bbfd556d746510aa35f20828c69c84022e907d3dfb7d76b2ba54256"} err="failed to get container status \"ac2f07ad2bbfd556d746510aa35f20828c69c84022e907d3dfb7d76b2ba54256\": rpc error: code = NotFound desc = could not find container \"ac2f07ad2bbfd556d746510aa35f20828c69c84022e907d3dfb7d76b2ba54256\": container with ID starting with ac2f07ad2bbfd556d746510aa35f20828c69c84022e907d3dfb7d76b2ba54256 not found: ID does not exist"
Jul 20 20:11:47.854564 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:47.854540    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftp55\" (UniqueName: \"kubernetes.io/projected/02d894ae-8ac2-47e8-a27e-d075bf99ea3d-kube-api-access-ftp55\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:11:47.854564 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:47.854561    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/02d894ae-8ac2-47e8-a27e-d075bf99ea3d-webhook-cert\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:11:47.857515 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:47.857495    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77bdf94859-pxtr7"]
Jul 20 20:11:47.864103 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:47.864083    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-77bdf94859-pxtr7"]
Jul 20 20:11:48.664823 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:48.664783    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77bdf94859-xg8jz"]
Jul 20 20:11:48.665221 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:48.665072    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02d894ae-8ac2-47e8-a27e-d075bf99ea3d" containerName="manager"
Jul 20 20:11:48.665221 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:48.665084    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="02d894ae-8ac2-47e8-a27e-d075bf99ea3d" containerName="manager"
Jul 20 20:11:48.665221 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:48.665142    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="02d894ae-8ac2-47e8-a27e-d075bf99ea3d" containerName="manager"
Jul 20 20:11:48.669174 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:48.669157    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77bdf94859-xg8jz"
Jul 20 20:11:48.673200 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:48.673178    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 20 20:11:48.673325 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:48.673239    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 20 20:11:48.673325 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:48.673242    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4qdn7\""
Jul 20 20:11:48.679239 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:48.679217    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77bdf94859-xg8jz"]
Jul 20 20:11:48.761263 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:48.761228    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/414f1fc4-6669-492b-9ee8-893b2088c3f0-webhook-cert\") pod \"maas-controller-77bdf94859-xg8jz\" (UID: \"414f1fc4-6669-492b-9ee8-893b2088c3f0\") " pod="opendatahub/maas-controller-77bdf94859-xg8jz"
Jul 20 20:11:48.761446 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:48.761271    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsnbb\" (UniqueName: \"kubernetes.io/projected/414f1fc4-6669-492b-9ee8-893b2088c3f0-kube-api-access-gsnbb\") pod \"maas-controller-77bdf94859-xg8jz\" (UID: \"414f1fc4-6669-492b-9ee8-893b2088c3f0\") " pod="opendatahub/maas-controller-77bdf94859-xg8jz"
Jul 20 20:11:48.861742 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:48.861712    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsnbb\" (UniqueName: \"kubernetes.io/projected/414f1fc4-6669-492b-9ee8-893b2088c3f0-kube-api-access-gsnbb\") pod \"maas-controller-77bdf94859-xg8jz\" (UID: \"414f1fc4-6669-492b-9ee8-893b2088c3f0\") " pod="opendatahub/maas-controller-77bdf94859-xg8jz"
Jul 20 20:11:48.861899 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:48.861809    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/414f1fc4-6669-492b-9ee8-893b2088c3f0-webhook-cert\") pod \"maas-controller-77bdf94859-xg8jz\" (UID: \"414f1fc4-6669-492b-9ee8-893b2088c3f0\") " pod="opendatahub/maas-controller-77bdf94859-xg8jz"
Jul 20 20:11:48.864219 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:48.864195    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/414f1fc4-6669-492b-9ee8-893b2088c3f0-webhook-cert\") pod \"maas-controller-77bdf94859-xg8jz\" (UID: \"414f1fc4-6669-492b-9ee8-893b2088c3f0\") " pod="opendatahub/maas-controller-77bdf94859-xg8jz"
Jul 20 20:11:48.871136 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:48.871106    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsnbb\" (UniqueName: \"kubernetes.io/projected/414f1fc4-6669-492b-9ee8-893b2088c3f0-kube-api-access-gsnbb\") pod \"maas-controller-77bdf94859-xg8jz\" (UID: \"414f1fc4-6669-492b-9ee8-893b2088c3f0\") " pod="opendatahub/maas-controller-77bdf94859-xg8jz"
Jul 20 20:11:48.979719 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:48.979609    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77bdf94859-xg8jz"
Jul 20 20:11:49.113323 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:49.113268    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77bdf94859-xg8jz"]
Jul 20 20:11:49.115749 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:11:49.115720    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod414f1fc4_6669_492b_9ee8_893b2088c3f0.slice/crio-2cb81bf64d88848eba052b223a35961495e7aacad2f86212f487ba59b39dc7a3 WatchSource:0}: Error finding container 2cb81bf64d88848eba052b223a35961495e7aacad2f86212f487ba59b39dc7a3: Status 404 returned error can't find the container with id 2cb81bf64d88848eba052b223a35961495e7aacad2f86212f487ba59b39dc7a3
Jul 20 20:11:49.176143 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:49.176115    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02d894ae-8ac2-47e8-a27e-d075bf99ea3d" path="/var/lib/kubelet/pods/02d894ae-8ac2-47e8-a27e-d075bf99ea3d/volumes"
Jul 20 20:11:49.846248 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:49.846214    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77bdf94859-xg8jz" event={"ID":"414f1fc4-6669-492b-9ee8-893b2088c3f0","Type":"ContainerStarted","Data":"37621a0e3ebf0beb145b9e4c803922034d932bba4787d6ceeef16873d7b53877"}
Jul 20 20:11:49.846611 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:49.846263    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77bdf94859-xg8jz" event={"ID":"414f1fc4-6669-492b-9ee8-893b2088c3f0","Type":"ContainerStarted","Data":"2cb81bf64d88848eba052b223a35961495e7aacad2f86212f487ba59b39dc7a3"}
Jul 20 20:11:49.846611 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:49.846340    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77bdf94859-xg8jz"
Jul 20 20:11:49.869002 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:11:49.868932    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77bdf94859-xg8jz" podStartSLOduration=1.476259199 podStartE2EDuration="1.868915825s" totalImagesPullingTime="392.656626ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:11:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:11:49.868349743 +0000 UTC m=+925.132419803" watchObservedRunningTime="2026-07-20 20:11:49.868915825 +0000 UTC m=+925.132985869"
Jul 20 20:12:00.852497 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:12:00.852464    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77bdf94859-xg8jz"
Jul 20 20:16:25.113874 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:16:25.113848    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 20:16:25.114509 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:16:25.114490    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 20:20:34.237239 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.237202    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"]
Jul 20 20:20:34.240344 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.240319    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:34.243212 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.243192    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-e0633ac2-kserve-self-signed-certs\""
Jul 20 20:20:34.258739 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.258706    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"]
Jul 20 20:20:34.309112 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.309076    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-model-cache\") pod \"e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") " pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:34.309289 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.309128    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxmk6\" (UniqueName: \"kubernetes.io/projected/374986e8-ce14-4151-a116-742b5f7d0fb0-kube-api-access-pxmk6\") pod \"e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") " pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:34.309289 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.309199    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/374986e8-ce14-4151-a116-742b5f7d0fb0-tls-certs\") pod \"e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") " pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:34.309289 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.309255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-tmp-dir\") pod \"e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") " pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:34.309289 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.309284    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-home\") pod \"e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") " pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:34.309427 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.309312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-dshm\") pod \"e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") " pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:34.410579 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.410541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-tmp-dir\") pod \"e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") " pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:34.410579 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.410578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-home\") pod \"e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") " pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:34.410811 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.410592    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-dshm\") pod \"e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") " pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:34.410811 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.410619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-model-cache\") pod \"e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") " pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:34.410811 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.410650    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxmk6\" (UniqueName: \"kubernetes.io/projected/374986e8-ce14-4151-a116-742b5f7d0fb0-kube-api-access-pxmk6\") pod \"e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") " pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:34.410811 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.410690    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/374986e8-ce14-4151-a116-742b5f7d0fb0-tls-certs\") pod \"e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") " pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:34.411051 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.410953    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-home\") pod \"e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") " pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:34.411112 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.411073    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-tmp-dir\") pod \"e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") " pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:34.411152 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.411112    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-model-cache\") pod \"e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") " pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:34.412954 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.412936    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-dshm\") pod \"e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") " pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:34.413212 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.413191    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/374986e8-ce14-4151-a116-742b5f7d0fb0-tls-certs\") pod \"e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") " pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:34.418714 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.418687    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxmk6\" (UniqueName: \"kubernetes.io/projected/374986e8-ce14-4151-a116-742b5f7d0fb0-kube-api-access-pxmk6\") pod \"e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") " pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:34.549739 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.549714    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:34.676095 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.676071    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"]
Jul 20 20:20:34.678799 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:20:34.678767    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod374986e8_ce14_4151_a116_742b5f7d0fb0.slice/crio-9ae7292337e6661d0e5cbb17513882717a7b96d0edfa56805581223e40aac979 WatchSource:0}: Error finding container 9ae7292337e6661d0e5cbb17513882717a7b96d0edfa56805581223e40aac979: Status 404 returned error can't find the container with id 9ae7292337e6661d0e5cbb17513882717a7b96d0edfa56805581223e40aac979
Jul 20 20:20:34.985066 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.984948    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w" event={"ID":"374986e8-ce14-4151-a116-742b5f7d0fb0","Type":"ContainerStarted","Data":"7b65ee125e97d0974923f24572de70ca7ea5b3f9ba38a13ff22146d3e4438510"}
Jul 20 20:20:34.985066 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:34.985012    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w" event={"ID":"374986e8-ce14-4151-a116-742b5f7d0fb0","Type":"ContainerStarted","Data":"9ae7292337e6661d0e5cbb17513882717a7b96d0edfa56805581223e40aac979"}
Jul 20 20:20:35.003604 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:35.003557    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w" podStartSLOduration=1.003543417 podStartE2EDuration="1.003543417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:20:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:20:35.001146504 +0000 UTC m=+1450.265216549" watchObservedRunningTime="2026-07-20 20:20:35.003543417 +0000 UTC m=+1450.267613463"
Jul 20 20:20:44.550504 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:44.550473    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:44.550934 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:44.550533    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:44.563027 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:44.562993    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:45.047368 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:45.047283    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:20:53.939974 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:53.939925    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"]
Jul 20 20:20:53.943297 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:53.943279    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:20:53.945934 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:53.945909    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-e0633ac2-kserve-self-signed-certs\""
Jul 20 20:20:53.951606 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:53.951583    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"]
Jul 20 20:20:54.077155 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:54.077129    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/561c069a-85f0-433b-94d5-d8782d724daf-tls-certs\") pod \"e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") " pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:20:54.077303 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:54.077168    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fn2j\" (UniqueName: \"kubernetes.io/projected/561c069a-85f0-433b-94d5-d8782d724daf-kube-api-access-7fn2j\") pod \"e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") " pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:20:54.077303 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:54.077226    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-home\") pod \"e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") " pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:20:54.077303 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:54.077267    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-dshm\") pod \"e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") " pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:20:54.077303 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:54.077284    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-tmp-dir\") pod \"e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") " pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:20:54.077478 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:54.077321    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-model-cache\") pod \"e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") " pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:20:54.178575 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:54.178540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-home\") pod \"e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") " pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:20:54.178734 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:54.178584    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-dshm\") pod \"e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") " pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:20:54.178734 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:54.178611    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-tmp-dir\") pod \"e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") " pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:20:54.178734 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:54.178653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-model-cache\") pod \"e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") " pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:20:54.178884 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:54.178822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/561c069a-85f0-433b-94d5-d8782d724daf-tls-certs\") pod \"e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") " pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:20:54.178936 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:54.178898    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fn2j\" (UniqueName: \"kubernetes.io/projected/561c069a-85f0-433b-94d5-d8782d724daf-kube-api-access-7fn2j\") pod \"e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") " pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:20:54.179090 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:54.179068    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-home\") pod \"e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") " pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:20:54.179191 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:54.179116    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-model-cache\") pod \"e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") " pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:20:54.179191 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:54.179172    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-tmp-dir\") pod \"e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") " pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:20:54.180834 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:54.180813    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-dshm\") pod \"e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") " pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:20:54.181169 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:54.181150    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/561c069a-85f0-433b-94d5-d8782d724daf-tls-certs\") pod \"e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") " pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:20:54.187193 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:54.187170    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fn2j\" (UniqueName: \"kubernetes.io/projected/561c069a-85f0-433b-94d5-d8782d724daf-kube-api-access-7fn2j\") pod \"e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") " pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:20:54.253233 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:54.253161    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:20:54.383909 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:54.383884    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"]
Jul 20 20:20:54.385993 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:20:54.385951    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod561c069a_85f0_433b_94d5_d8782d724daf.slice/crio-980462f391773be34738b4ee09b9c04693f6f239b618f84a818cfcdfe9513d2b WatchSource:0}: Error finding container 980462f391773be34738b4ee09b9c04693f6f239b618f84a818cfcdfe9513d2b: Status 404 returned error can't find the container with id 980462f391773be34738b4ee09b9c04693f6f239b618f84a818cfcdfe9513d2b
Jul 20 20:20:55.079669 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:55.079625    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725" event={"ID":"561c069a-85f0-433b-94d5-d8782d724daf","Type":"ContainerStarted","Data":"6564cdf005254ff49ea34eb2f2390a8d17a03f54707298876b6f4471768c9754"}
Jul 20 20:20:55.079669 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:55.079669    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725" event={"ID":"561c069a-85f0-433b-94d5-d8782d724daf","Type":"ContainerStarted","Data":"980462f391773be34738b4ee09b9c04693f6f239b618f84a818cfcdfe9513d2b"}
Jul 20 20:20:55.098874 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:20:55.098829    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725" podStartSLOduration=2.09881477 podStartE2EDuration="2.09881477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:20:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:20:55.097159671 +0000 UTC m=+1470.361229716" watchObservedRunningTime="2026-07-20 20:20:55.09881477 +0000 UTC m=+1470.362884815"
Jul 20 20:21:04.253770 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:04.253727    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:21:04.254261 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:04.253790    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:21:04.266359 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:04.266327    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:21:05.132769 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:05.132737    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:21:14.152318 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:14.152286    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"]
Jul 20 20:21:15.341577 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:15.341544    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"]
Jul 20 20:21:25.136614 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:25.136585    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 20:21:25.139766 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:25.139743    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 20:21:29.173668 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:29.173624    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w" podUID="374986e8-ce14-4151-a116-742b5f7d0fb0" containerName="main" containerID="cri-o://7b65ee125e97d0974923f24572de70ca7ea5b3f9ba38a13ff22146d3e4438510" gracePeriod=45
Jul 20 20:21:29.431476 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:29.431421    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:21:29.484441 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:29.484411    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-tmp-dir\") pod \"374986e8-ce14-4151-a116-742b5f7d0fb0\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") "
Jul 20 20:21:29.484441 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:29.484440    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-home\") pod \"374986e8-ce14-4151-a116-742b5f7d0fb0\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") "
Jul 20 20:21:29.484631 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:29.484487    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-model-cache\") pod \"374986e8-ce14-4151-a116-742b5f7d0fb0\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") "
Jul 20 20:21:29.484631 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:29.484535    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/374986e8-ce14-4151-a116-742b5f7d0fb0-tls-certs\") pod \"374986e8-ce14-4151-a116-742b5f7d0fb0\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") "
Jul 20 20:21:29.484631 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:29.484559    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-dshm\") pod \"374986e8-ce14-4151-a116-742b5f7d0fb0\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") "
Jul 20 20:21:29.484631 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:29.484594    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxmk6\" (UniqueName: \"kubernetes.io/projected/374986e8-ce14-4151-a116-742b5f7d0fb0-kube-api-access-pxmk6\") pod \"374986e8-ce14-4151-a116-742b5f7d0fb0\" (UID: \"374986e8-ce14-4151-a116-742b5f7d0fb0\") "
Jul 20 20:21:29.484834 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:29.484734    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "374986e8-ce14-4151-a116-742b5f7d0fb0" (UID: "374986e8-ce14-4151-a116-742b5f7d0fb0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:21:29.484834 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:29.484744    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-model-cache" (OuterVolumeSpecName: "model-cache") pod "374986e8-ce14-4151-a116-742b5f7d0fb0" (UID: "374986e8-ce14-4151-a116-742b5f7d0fb0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:21:29.484834 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:29.484769    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-home" (OuterVolumeSpecName: "home") pod "374986e8-ce14-4151-a116-742b5f7d0fb0" (UID: "374986e8-ce14-4151-a116-742b5f7d0fb0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:21:29.485104 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:29.484915    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-tmp-dir\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:21:29.485104 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:29.484936    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-home\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:21:29.485104 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:29.484954    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-model-cache\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:21:29.486766 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:29.486741    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-dshm" (OuterVolumeSpecName: "dshm") pod "374986e8-ce14-4151-a116-742b5f7d0fb0" (UID: "374986e8-ce14-4151-a116-742b5f7d0fb0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:21:29.486872 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:29.486837    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/374986e8-ce14-4151-a116-742b5f7d0fb0-kube-api-access-pxmk6" (OuterVolumeSpecName: "kube-api-access-pxmk6") pod "374986e8-ce14-4151-a116-742b5f7d0fb0" (UID: "374986e8-ce14-4151-a116-742b5f7d0fb0"). InnerVolumeSpecName "kube-api-access-pxmk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:21:29.486913 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:29.486877    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/374986e8-ce14-4151-a116-742b5f7d0fb0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "374986e8-ce14-4151-a116-742b5f7d0fb0" (UID: "374986e8-ce14-4151-a116-742b5f7d0fb0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 20:21:29.586161 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:29.586130    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/374986e8-ce14-4151-a116-742b5f7d0fb0-tls-certs\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:21:29.586161 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:29.586155    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/374986e8-ce14-4151-a116-742b5f7d0fb0-dshm\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:21:29.586161 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:29.586164    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxmk6\" (UniqueName: \"kubernetes.io/projected/374986e8-ce14-4151-a116-742b5f7d0fb0-kube-api-access-pxmk6\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:21:30.225145 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.225108    2580 generic.go:356] "Generic (PLEG): container finished" podID="374986e8-ce14-4151-a116-742b5f7d0fb0" containerID="7b65ee125e97d0974923f24572de70ca7ea5b3f9ba38a13ff22146d3e4438510" exitCode=0
Jul 20 20:21:30.225145 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.225146    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"
Jul 20 20:21:30.225608 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.225146    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w" event={"ID":"374986e8-ce14-4151-a116-742b5f7d0fb0","Type":"ContainerDied","Data":"7b65ee125e97d0974923f24572de70ca7ea5b3f9ba38a13ff22146d3e4438510"}
Jul 20 20:21:30.225608 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.225277    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w" event={"ID":"374986e8-ce14-4151-a116-742b5f7d0fb0","Type":"ContainerDied","Data":"9ae7292337e6661d0e5cbb17513882717a7b96d0edfa56805581223e40aac979"}
Jul 20 20:21:30.225608 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.225303    2580 scope.go:117] "RemoveContainer" containerID="7b65ee125e97d0974923f24572de70ca7ea5b3f9ba38a13ff22146d3e4438510"
Jul 20 20:21:30.234526 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.234512    2580 scope.go:117] "RemoveContainer" containerID="7b65ee125e97d0974923f24572de70ca7ea5b3f9ba38a13ff22146d3e4438510"
Jul 20 20:21:30.234766 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:21:30.234750    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b65ee125e97d0974923f24572de70ca7ea5b3f9ba38a13ff22146d3e4438510\": container with ID starting with 7b65ee125e97d0974923f24572de70ca7ea5b3f9ba38a13ff22146d3e4438510 not found: ID does not exist" containerID="7b65ee125e97d0974923f24572de70ca7ea5b3f9ba38a13ff22146d3e4438510"
Jul 20 20:21:30.234813 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.234774    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b65ee125e97d0974923f24572de70ca7ea5b3f9ba38a13ff22146d3e4438510"} err="failed to get container status \"7b65ee125e97d0974923f24572de70ca7ea5b3f9ba38a13ff22146d3e4438510\": rpc error: code = NotFound desc = could not find container \"7b65ee125e97d0974923f24572de70ca7ea5b3f9ba38a13ff22146d3e4438510\": container with ID starting with 7b65ee125e97d0974923f24572de70ca7ea5b3f9ba38a13ff22146d3e4438510 not found: ID does not exist"
Jul 20 20:21:30.246927 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.246905    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"]
Jul 20 20:21:30.252815 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.252789    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-e0633ac2-kserve-7f76959687-q2m7w"]
Jul 20 20:21:30.361384 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.361341    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725" podUID="561c069a-85f0-433b-94d5-d8782d724daf" containerName="main" containerID="cri-o://6564cdf005254ff49ea34eb2f2390a8d17a03f54707298876b6f4471768c9754" gracePeriod=45
Jul 20 20:21:30.611192 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.611166    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:21:30.694003 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.693946    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fn2j\" (UniqueName: \"kubernetes.io/projected/561c069a-85f0-433b-94d5-d8782d724daf-kube-api-access-7fn2j\") pod \"561c069a-85f0-433b-94d5-d8782d724daf\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") "
Jul 20 20:21:30.694180 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.694017    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/561c069a-85f0-433b-94d5-d8782d724daf-tls-certs\") pod \"561c069a-85f0-433b-94d5-d8782d724daf\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") "
Jul 20 20:21:30.694180 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.694038    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-model-cache\") pod \"561c069a-85f0-433b-94d5-d8782d724daf\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") "
Jul 20 20:21:30.694180 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.694067    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-dshm\") pod \"561c069a-85f0-433b-94d5-d8782d724daf\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") "
Jul 20 20:21:30.694180 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.694096    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-tmp-dir\") pod \"561c069a-85f0-433b-94d5-d8782d724daf\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") "
Jul 20 20:21:30.694180 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.694132    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-home\") pod \"561c069a-85f0-433b-94d5-d8782d724daf\" (UID: \"561c069a-85f0-433b-94d5-d8782d724daf\") "
Jul 20 20:21:30.694441 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.694345    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-model-cache" (OuterVolumeSpecName: "model-cache") pod "561c069a-85f0-433b-94d5-d8782d724daf" (UID: "561c069a-85f0-433b-94d5-d8782d724daf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:21:30.694441 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.694357    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "561c069a-85f0-433b-94d5-d8782d724daf" (UID: "561c069a-85f0-433b-94d5-d8782d724daf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:21:30.694534 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.694436    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-home" (OuterVolumeSpecName: "home") pod "561c069a-85f0-433b-94d5-d8782d724daf" (UID: "561c069a-85f0-433b-94d5-d8782d724daf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:21:30.696123 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.696100    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-dshm" (OuterVolumeSpecName: "dshm") pod "561c069a-85f0-433b-94d5-d8782d724daf" (UID: "561c069a-85f0-433b-94d5-d8782d724daf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:21:30.696221 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.696134    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/561c069a-85f0-433b-94d5-d8782d724daf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "561c069a-85f0-433b-94d5-d8782d724daf" (UID: "561c069a-85f0-433b-94d5-d8782d724daf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 20:21:30.696221 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.696188    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/561c069a-85f0-433b-94d5-d8782d724daf-kube-api-access-7fn2j" (OuterVolumeSpecName: "kube-api-access-7fn2j") pod "561c069a-85f0-433b-94d5-d8782d724daf" (UID: "561c069a-85f0-433b-94d5-d8782d724daf"). InnerVolumeSpecName "kube-api-access-7fn2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:21:30.795689 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.795612    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-home\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:21:30.795689 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.795640    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fn2j\" (UniqueName: \"kubernetes.io/projected/561c069a-85f0-433b-94d5-d8782d724daf-kube-api-access-7fn2j\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:21:30.795689 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.795653    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/561c069a-85f0-433b-94d5-d8782d724daf-tls-certs\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:21:30.795689 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.795671    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-model-cache\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:21:30.795689 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.795679    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-dshm\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:21:30.795689 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:30.795686    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/561c069a-85f0-433b-94d5-d8782d724daf-tmp-dir\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:21:31.175382 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:31.175349    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="374986e8-ce14-4151-a116-742b5f7d0fb0" path="/var/lib/kubelet/pods/374986e8-ce14-4151-a116-742b5f7d0fb0/volumes"
Jul 20 20:21:31.230131 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:31.230099    2580 generic.go:356] "Generic (PLEG): container finished" podID="561c069a-85f0-433b-94d5-d8782d724daf" containerID="6564cdf005254ff49ea34eb2f2390a8d17a03f54707298876b6f4471768c9754" exitCode=0
Jul 20 20:21:31.230576 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:31.230149    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"
Jul 20 20:21:31.230576 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:31.230189    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725" event={"ID":"561c069a-85f0-433b-94d5-d8782d724daf","Type":"ContainerDied","Data":"6564cdf005254ff49ea34eb2f2390a8d17a03f54707298876b6f4471768c9754"}
Jul 20 20:21:31.230576 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:31.230227    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725" event={"ID":"561c069a-85f0-433b-94d5-d8782d724daf","Type":"ContainerDied","Data":"980462f391773be34738b4ee09b9c04693f6f239b618f84a818cfcdfe9513d2b"}
Jul 20 20:21:31.230576 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:31.230244    2580 scope.go:117] "RemoveContainer" containerID="6564cdf005254ff49ea34eb2f2390a8d17a03f54707298876b6f4471768c9754"
Jul 20 20:21:31.243816 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:31.243773    2580 scope.go:117] "RemoveContainer" containerID="6564cdf005254ff49ea34eb2f2390a8d17a03f54707298876b6f4471768c9754"
Jul 20 20:21:31.244468 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:21:31.244440    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6564cdf005254ff49ea34eb2f2390a8d17a03f54707298876b6f4471768c9754\": container with ID starting with 6564cdf005254ff49ea34eb2f2390a8d17a03f54707298876b6f4471768c9754 not found: ID does not exist" containerID="6564cdf005254ff49ea34eb2f2390a8d17a03f54707298876b6f4471768c9754"
Jul 20 20:21:31.244534 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:31.244481    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6564cdf005254ff49ea34eb2f2390a8d17a03f54707298876b6f4471768c9754"} err="failed to get container status \"6564cdf005254ff49ea34eb2f2390a8d17a03f54707298876b6f4471768c9754\": rpc error: code = NotFound desc = could not find container \"6564cdf005254ff49ea34eb2f2390a8d17a03f54707298876b6f4471768c9754\": container with ID starting with 6564cdf005254ff49ea34eb2f2390a8d17a03f54707298876b6f4471768c9754 not found: ID does not exist"
Jul 20 20:21:31.252790 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:31.252766    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"]
Jul 20 20:21:31.256670 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:31.256645    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-e0633ac2-kserve-7fdd5f6d79-d7725"]
Jul 20 20:21:33.176203 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:21:33.176172    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="561c069a-85f0-433b-94d5-d8782d724daf" path="/var/lib/kubelet/pods/561c069a-85f0-433b-94d5-d8782d724daf/volumes"
Jul 20 20:25:36.501553 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:36.501515    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d5e1ae9-kf95n"]
Jul 20 20:25:36.501999 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:36.501802    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="374986e8-ce14-4151-a116-742b5f7d0fb0" containerName="main"
Jul 20 20:25:36.501999 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:36.501814    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="374986e8-ce14-4151-a116-742b5f7d0fb0" containerName="main"
Jul 20 20:25:36.501999 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:36.501851    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="561c069a-85f0-433b-94d5-d8782d724daf" containerName="main"
Jul 20 20:25:36.501999 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:36.501857    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="561c069a-85f0-433b-94d5-d8782d724daf" containerName="main"
Jul 20 20:25:36.501999 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:36.501909    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="374986e8-ce14-4151-a116-742b5f7d0fb0" containerName="main"
Jul 20 20:25:36.501999 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:36.501917    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="561c069a-85f0-433b-94d5-d8782d724daf" containerName="main"
Jul 20 20:25:36.504708 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:36.504692    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d5e1ae9-kf95n"
Jul 20 20:25:36.507721 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:36.507701    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-628hb\""
Jul 20 20:25:36.507835 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:36.507784    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 20:25:36.515630 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:36.515608    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d5e1ae9-kf95n"]
Jul 20 20:25:36.556316 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:36.556282    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06324f42-d29b-4eb5-be9c-93da8aff77c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8d5e1ae9-kf95n\" (UID: \"06324f42-d29b-4eb5-be9c-93da8aff77c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d5e1ae9-kf95n"
Jul 20 20:25:36.656719 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:36.656685    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06324f42-d29b-4eb5-be9c-93da8aff77c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8d5e1ae9-kf95n\" (UID: \"06324f42-d29b-4eb5-be9c-93da8aff77c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d5e1ae9-kf95n"
Jul 20 20:25:36.657350 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:36.657330    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06324f42-d29b-4eb5-be9c-93da8aff77c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8d5e1ae9-kf95n\" (UID: \"06324f42-d29b-4eb5-be9c-93da8aff77c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d5e1ae9-kf95n"
Jul 20 20:25:36.814582 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:36.814553    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d5e1ae9-kf95n"
Jul 20 20:25:36.952920 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:36.952842    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d5e1ae9-kf95n"]
Jul 20 20:25:36.955858 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:25:36.955827    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06324f42_d29b_4eb5_be9c_93da8aff77c6.slice/crio-d9c1aab28697080747fae6dc33a1d9792ace94d00f0a6bddd2b15678446ec21a WatchSource:0}: Error finding container d9c1aab28697080747fae6dc33a1d9792ace94d00f0a6bddd2b15678446ec21a: Status 404 returned error can't find the container with id d9c1aab28697080747fae6dc33a1d9792ace94d00f0a6bddd2b15678446ec21a
Jul 20 20:25:36.957802 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:36.957789    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 20:25:37.227666 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:37.227578    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d5e1ae9-kf95n" event={"ID":"06324f42-d29b-4eb5-be9c-93da8aff77c6","Type":"ContainerStarted","Data":"d9c1aab28697080747fae6dc33a1d9792ace94d00f0a6bddd2b15678446ec21a"}
Jul 20 20:25:40.241412 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:40.241324    2580 generic.go:356] "Generic (PLEG): container finished" podID="06324f42-d29b-4eb5-be9c-93da8aff77c6" containerID="e1b0d5bbce45811b26708e6ef00635f62e71e4142d8c2fe61a8da4c6ee7ef620" exitCode=0
Jul 20 20:25:40.241775 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:40.241404    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d5e1ae9-kf95n" event={"ID":"06324f42-d29b-4eb5-be9c-93da8aff77c6","Type":"ContainerDied","Data":"e1b0d5bbce45811b26708e6ef00635f62e71e4142d8c2fe61a8da4c6ee7ef620"}
Jul 20 20:25:41.369761 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:41.369737    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d5e1ae9-kf95n"
Jul 20 20:25:41.389361 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:41.389336    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06324f42-d29b-4eb5-be9c-93da8aff77c6-maas-api-service-ca\") pod \"06324f42-d29b-4eb5-be9c-93da8aff77c6\" (UID: \"06324f42-d29b-4eb5-be9c-93da8aff77c6\") "
Jul 20 20:25:41.389767 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:41.389742    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06324f42-d29b-4eb5-be9c-93da8aff77c6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "06324f42-d29b-4eb5-be9c-93da8aff77c6" (UID: "06324f42-d29b-4eb5-be9c-93da8aff77c6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 20:25:41.490565 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:41.490526    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06324f42-d29b-4eb5-be9c-93da8aff77c6-maas-api-service-ca\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:25:42.249883 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:42.249850    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d5e1ae9-kf95n" event={"ID":"06324f42-d29b-4eb5-be9c-93da8aff77c6","Type":"ContainerDied","Data":"d9c1aab28697080747fae6dc33a1d9792ace94d00f0a6bddd2b15678446ec21a"}
Jul 20 20:25:42.250070 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:42.249898    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d9c1aab28697080747fae6dc33a1d9792ace94d00f0a6bddd2b15678446ec21a"
Jul 20 20:25:42.250070 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:42.249864    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d5e1ae9-kf95n"
Jul 20 20:25:51.585344 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:51.585303    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d5e1ae9-kf95n"]
Jul 20 20:25:51.591329 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:51.591303    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8d5e1ae9-kf95n"]
Jul 20 20:25:53.175258 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:25:53.175224    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06324f42-d29b-4eb5-be9c-93da8aff77c6" path="/var/lib/kubelet/pods/06324f42-d29b-4eb5-be9c-93da8aff77c6/volumes"
Jul 20 20:26:20.438081 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:20.438042    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-03a646a1-8splj"]
Jul 20 20:26:20.438506 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:20.438489    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06324f42-d29b-4eb5-be9c-93da8aff77c6" containerName="revoke-keys"
Jul 20 20:26:20.438549 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:20.438510    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="06324f42-d29b-4eb5-be9c-93da8aff77c6" containerName="revoke-keys"
Jul 20 20:26:20.438617 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:20.438606    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="06324f42-d29b-4eb5-be9c-93da8aff77c6" containerName="revoke-keys"
Jul 20 20:26:20.446301 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:20.446276    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-03a646a1-8splj"
Jul 20 20:26:20.449547 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:20.449523    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 20:26:20.449937 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:20.449916    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-628hb\""
Jul 20 20:26:20.450503 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:20.450480    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-03a646a1-8splj"]
Jul 20 20:26:20.508979 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:20.508926    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8a750ed-faca-4cbb-9b42-7f02df15ccb5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-03a646a1-8splj\" (UID: \"c8a750ed-faca-4cbb-9b42-7f02df15ccb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-03a646a1-8splj"
Jul 20 20:26:20.609915 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:20.609878    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8a750ed-faca-4cbb-9b42-7f02df15ccb5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-03a646a1-8splj\" (UID: \"c8a750ed-faca-4cbb-9b42-7f02df15ccb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-03a646a1-8splj"
Jul 20 20:26:20.610584 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:20.610560    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8a750ed-faca-4cbb-9b42-7f02df15ccb5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-03a646a1-8splj\" (UID: \"c8a750ed-faca-4cbb-9b42-7f02df15ccb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-03a646a1-8splj"
Jul 20 20:26:20.757185 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:20.757102    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-03a646a1-8splj"
Jul 20 20:26:20.890773 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:20.890745    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-03a646a1-8splj"]
Jul 20 20:26:20.892367 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:26:20.892332    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8a750ed_faca_4cbb_9b42_7f02df15ccb5.slice/crio-6e9dc3c5fd522d1a48539bf48422a45eeb3db22559a24d97770876c988e7f3c7 WatchSource:0}: Error finding container 6e9dc3c5fd522d1a48539bf48422a45eeb3db22559a24d97770876c988e7f3c7: Status 404 returned error can't find the container with id 6e9dc3c5fd522d1a48539bf48422a45eeb3db22559a24d97770876c988e7f3c7
Jul 20 20:26:21.411155 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:21.411073    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-03a646a1-8splj" event={"ID":"c8a750ed-faca-4cbb-9b42-7f02df15ccb5","Type":"ContainerStarted","Data":"df8680503632afba12e8c29c304a8ea622090735852cf2520f9667d861291847"}
Jul 20 20:26:21.411155 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:21.411130    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-03a646a1-8splj" event={"ID":"c8a750ed-faca-4cbb-9b42-7f02df15ccb5","Type":"ContainerStarted","Data":"6e9dc3c5fd522d1a48539bf48422a45eeb3db22559a24d97770876c988e7f3c7"}
Jul 20 20:26:21.444294 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:21.444235    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-03a646a1-8splj" podStartSLOduration=1.444214193 podStartE2EDuration="1.444214193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:26:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:26:21.444209015 +0000 UTC m=+1796.708279062" watchObservedRunningTime="2026-07-20 20:26:21.444214193 +0000 UTC m=+1796.708284239"
Jul 20 20:26:22.416667 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:22.416628    2580 generic.go:356] "Generic (PLEG): container finished" podID="c8a750ed-faca-4cbb-9b42-7f02df15ccb5" containerID="df8680503632afba12e8c29c304a8ea622090735852cf2520f9667d861291847" exitCode=0
Jul 20 20:26:22.416952 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:22.416680    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-03a646a1-8splj" event={"ID":"c8a750ed-faca-4cbb-9b42-7f02df15ccb5","Type":"ContainerDied","Data":"df8680503632afba12e8c29c304a8ea622090735852cf2520f9667d861291847"}
Jul 20 20:26:23.544027 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:23.544004    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-03a646a1-8splj"
Jul 20 20:26:23.636408 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:23.636384    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8a750ed-faca-4cbb-9b42-7f02df15ccb5-maas-api-service-ca\") pod \"c8a750ed-faca-4cbb-9b42-7f02df15ccb5\" (UID: \"c8a750ed-faca-4cbb-9b42-7f02df15ccb5\") "
Jul 20 20:26:23.636720 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:23.636698    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8a750ed-faca-4cbb-9b42-7f02df15ccb5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c8a750ed-faca-4cbb-9b42-7f02df15ccb5" (UID: "c8a750ed-faca-4cbb-9b42-7f02df15ccb5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 20:26:23.736831 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:23.736768    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8a750ed-faca-4cbb-9b42-7f02df15ccb5-maas-api-service-ca\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:26:24.425246 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:24.425214    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-03a646a1-8splj"
Jul 20 20:26:24.425414 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:24.425214    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-03a646a1-8splj" event={"ID":"c8a750ed-faca-4cbb-9b42-7f02df15ccb5","Type":"ContainerDied","Data":"6e9dc3c5fd522d1a48539bf48422a45eeb3db22559a24d97770876c988e7f3c7"}
Jul 20 20:26:24.425414 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:24.425347    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e9dc3c5fd522d1a48539bf48422a45eeb3db22559a24d97770876c988e7f3c7"
Jul 20 20:26:25.158646 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:25.158620    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 20:26:25.162994 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:25.162971    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 20:26:35.560343 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:35.560310    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-03a646a1-8splj"]
Jul 20 20:26:35.563902 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:35.563879    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-03a646a1-8splj"]
Jul 20 20:26:36.629841 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.629809    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"]
Jul 20 20:26:36.630232 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.630122    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8a750ed-faca-4cbb-9b42-7f02df15ccb5" containerName="revoke-keys"
Jul 20 20:26:36.630232 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.630134    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8a750ed-faca-4cbb-9b42-7f02df15ccb5" containerName="revoke-keys"
Jul 20 20:26:36.630232 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.630191    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8a750ed-faca-4cbb-9b42-7f02df15ccb5" containerName="revoke-keys"
Jul 20 20:26:36.632855 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.632838    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.635866 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.635841    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 20 20:26:36.636148 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.636132    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-47f7982b-openshift-default-dockercfg-rq7nd\""
Jul 20 20:26:36.650632 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.650607    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"]
Jul 20 20:26:36.747470 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.747440    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-data\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.747627 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.747489    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfmjc\" (UniqueName: \"kubernetes.io/projected/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-kube-api-access-nfmjc\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.747627 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.747525    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-credential-socket\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.747627 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.747549    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istiod-ca-cert\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.747627 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.747575    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-workload-certs\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.747627 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.747617    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-podinfo\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.747788 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.747642    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-token\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.747788 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.747660    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-envoy\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.747788 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.747677    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-workload-socket\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.848839 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.848805    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istiod-ca-cert\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.848839 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.848844    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-workload-certs\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.849062 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.848886    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-podinfo\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.849062 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.848927    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-token\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.849062 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.848952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-envoy\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.849062 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.849001    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-workload-socket\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.849062 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.849038    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-data\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.849396 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.849369    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-workload-certs\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.849528 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.849389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfmjc\" (UniqueName: \"kubernetes.io/projected/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-kube-api-access-nfmjc\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.849528 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.849465    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-credential-socket\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.849528 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.849482    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-workload-socket\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.849528 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.849517    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-data\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.849722 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.849667    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istiod-ca-cert\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.849758 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.849733    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-credential-socket\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.851858 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.851832    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-envoy\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.852094 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.852079    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-podinfo\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.857833 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.857804    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-token\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.858952 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.858930    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfmjc\" (UniqueName: \"kubernetes.io/projected/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-kube-api-access-nfmjc\") pod \"e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") " pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:36.943976 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:36.943870    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:37.081748 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:37.081714    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"]
Jul 20 20:26:37.085053 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:26:37.085014    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda93842cd_5ef7_44b2_9e63_e0863e17d4d0.slice/crio-b26b9d252ca1b45477f3d0cf7726e87a4e2d8b116ccc63f7f9c064b6e8724b36 WatchSource:0}: Error finding container b26b9d252ca1b45477f3d0cf7726e87a4e2d8b116ccc63f7f9c064b6e8724b36: Status 404 returned error can't find the container with id b26b9d252ca1b45477f3d0cf7726e87a4e2d8b116ccc63f7f9c064b6e8724b36
Jul 20 20:26:37.175645 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:37.175614    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8a750ed-faca-4cbb-9b42-7f02df15ccb5" path="/var/lib/kubelet/pods/c8a750ed-faca-4cbb-9b42-7f02df15ccb5/volumes"
Jul 20 20:26:37.476937 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:37.476904    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv" event={"ID":"a93842cd-5ef7-44b2-9e63-e0863e17d4d0","Type":"ContainerStarted","Data":"b26b9d252ca1b45477f3d0cf7726e87a4e2d8b116ccc63f7f9c064b6e8724b36"}
Jul 20 20:26:40.036122 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:40.036082    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:26:40.036400 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:40.036158    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:26:40.036400 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:40.036186    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:26:40.493132 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:40.493100    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv" event={"ID":"a93842cd-5ef7-44b2-9e63-e0863e17d4d0","Type":"ContainerStarted","Data":"145bc6485ea47f58903270510db27d1c2b98170e3ec653227886ed271b4ab59d"}
Jul 20 20:26:40.518614 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:40.518564    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv" podStartSLOduration=1.569707178 podStartE2EDuration="4.518548855s" totalImagesPullingTime="2.948841677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:26:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:26:40.516637532 +0000 UTC m=+1815.780707578" watchObservedRunningTime="2026-07-20 20:26:40.518548855 +0000 UTC m=+1815.782618901"
Jul 20 20:26:40.944913 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:40.944861    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:41.949688 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:41.949658    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:42.502167 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:42.502136    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:42.503110 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:42.503095    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:54.553495 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:54.553458    2580 generic.go:356] "Generic (PLEG): container finished" podID="a93842cd-5ef7-44b2-9e63-e0863e17d4d0" containerID="145bc6485ea47f58903270510db27d1c2b98170e3ec653227886ed271b4ab59d" exitCode=0
Jul 20 20:26:54.553953 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:54.553531    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv" event={"ID":"a93842cd-5ef7-44b2-9e63-e0863e17d4d0","Type":"ContainerDied","Data":"145bc6485ea47f58903270510db27d1c2b98170e3ec653227886ed271b4ab59d"}
Jul 20 20:26:54.553953 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:54.553937    2580 scope.go:117] "RemoveContainer" containerID="145bc6485ea47f58903270510db27d1c2b98170e3ec653227886ed271b4ab59d"
Jul 20 20:26:54.555953 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:54.555915    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:26:54.556060 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:54.556043    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:26:54.556102 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:54.556089    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:26:55.558659 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:55.558622    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv" event={"ID":"a93842cd-5ef7-44b2-9e63-e0863e17d4d0","Type":"ContainerStarted","Data":"4646376a5107061a59a5098e0746f16aae1f86dc1f697f13ee075e1539b9d9d7"}
Jul 20 20:26:55.944768 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:55.944673    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:55.945650 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:55.945627    2580 patch_prober.go:28] interesting pod/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Jul 20 20:26:55.945714 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:55.945683    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv" podUID="a93842cd-5ef7-44b2-9e63-e0863e17d4d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Jul 20 20:26:56.944873 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:56.944837    2580 patch_prober.go:28] interesting pod/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Jul 20 20:26:56.945268 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:56.944897    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv" podUID="a93842cd-5ef7-44b2-9e63-e0863e17d4d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Jul 20 20:26:57.502076 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:57.502039    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:57.944585 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:57.944548    2580 patch_prober.go:28] interesting pod/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Jul 20 20:26:57.944756 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:57.944603    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv" podUID="a93842cd-5ef7-44b2-9e63-e0863e17d4d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Jul 20 20:26:58.944931 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:58.944883    2580 patch_prober.go:28] interesting pod/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Jul 20 20:26:58.945361 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:58.944946    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv" podUID="a93842cd-5ef7-44b2-9e63-e0863e17d4d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Jul 20 20:26:59.947979 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:59.947925    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:26:59.948998 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:26:59.948978    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:27:07.893146 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:07.893108    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47f7982b-26q7q"]
Jul 20 20:27:07.897330 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:07.897308    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47f7982b-26q7q"
Jul 20 20:27:07.900565 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:07.900540    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-628hb\""
Jul 20 20:27:07.900665 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:07.900594    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 20:27:07.904311 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:07.904290    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47f7982b-26q7q"]
Jul 20 20:27:08.024866 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:08.024825    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59abb16f-5c4e-433f-9b0a-d93c1bfb64a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-47f7982b-26q7q\" (UID: \"59abb16f-5c4e-433f-9b0a-d93c1bfb64a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47f7982b-26q7q"
Jul 20 20:27:08.126243 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:08.126212    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59abb16f-5c4e-433f-9b0a-d93c1bfb64a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-47f7982b-26q7q\" (UID: \"59abb16f-5c4e-433f-9b0a-d93c1bfb64a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47f7982b-26q7q"
Jul 20 20:27:08.126919 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:08.126898    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59abb16f-5c4e-433f-9b0a-d93c1bfb64a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-47f7982b-26q7q\" (UID: \"59abb16f-5c4e-433f-9b0a-d93c1bfb64a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47f7982b-26q7q"
Jul 20 20:27:08.208749 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:08.208655    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47f7982b-26q7q"
Jul 20 20:27:08.341802 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:08.341780    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47f7982b-26q7q"]
Jul 20 20:27:08.344002 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:27:08.343976    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59abb16f_5c4e_433f_9b0a_d93c1bfb64a0.slice/crio-50e71237e6ecd3925bb2b347823c2caefc5c26ff5646f3c7b2d6ff16338a107d WatchSource:0}: Error finding container 50e71237e6ecd3925bb2b347823c2caefc5c26ff5646f3c7b2d6ff16338a107d: Status 404 returned error can't find the container with id 50e71237e6ecd3925bb2b347823c2caefc5c26ff5646f3c7b2d6ff16338a107d
Jul 20 20:27:08.615678 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:08.615642    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47f7982b-26q7q" event={"ID":"59abb16f-5c4e-433f-9b0a-d93c1bfb64a0","Type":"ContainerStarted","Data":"50e71237e6ecd3925bb2b347823c2caefc5c26ff5646f3c7b2d6ff16338a107d"}
Jul 20 20:27:09.620970 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:09.620933    2580 generic.go:356] "Generic (PLEG): container finished" podID="59abb16f-5c4e-433f-9b0a-d93c1bfb64a0" containerID="202fdcad790ffbf6ad6f857f1a3393a96662aa0735f41bd74faa9a1530b47e2b" exitCode=0
Jul 20 20:27:09.621403 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:09.621015    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47f7982b-26q7q" event={"ID":"59abb16f-5c4e-433f-9b0a-d93c1bfb64a0","Type":"ContainerDied","Data":"202fdcad790ffbf6ad6f857f1a3393a96662aa0735f41bd74faa9a1530b47e2b"}
Jul 20 20:27:10.751325 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:10.751294    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47f7982b-26q7q"
Jul 20 20:27:10.852127 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:10.852096    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59abb16f-5c4e-433f-9b0a-d93c1bfb64a0-maas-api-service-ca\") pod \"59abb16f-5c4e-433f-9b0a-d93c1bfb64a0\" (UID: \"59abb16f-5c4e-433f-9b0a-d93c1bfb64a0\") "
Jul 20 20:27:10.852420 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:10.852400    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59abb16f-5c4e-433f-9b0a-d93c1bfb64a0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "59abb16f-5c4e-433f-9b0a-d93c1bfb64a0" (UID: "59abb16f-5c4e-433f-9b0a-d93c1bfb64a0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 20:27:10.953281 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:10.953202    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59abb16f-5c4e-433f-9b0a-d93c1bfb64a0-maas-api-service-ca\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:27:11.630742 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:11.630713    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47f7982b-26q7q"
Jul 20 20:27:11.630742 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:11.630722    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47f7982b-26q7q" event={"ID":"59abb16f-5c4e-433f-9b0a-d93c1bfb64a0","Type":"ContainerDied","Data":"50e71237e6ecd3925bb2b347823c2caefc5c26ff5646f3c7b2d6ff16338a107d"}
Jul 20 20:27:11.630945 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:11.630774    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50e71237e6ecd3925bb2b347823c2caefc5c26ff5646f3c7b2d6ff16338a107d"
Jul 20 20:27:22.980686 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:22.980649    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47f7982b-26q7q"]
Jul 20 20:27:22.984503 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:22.984479    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47f7982b-26q7q"]
Jul 20 20:27:23.175046 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:23.175010    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59abb16f-5c4e-433f-9b0a-d93c1bfb64a0" path="/var/lib/kubelet/pods/59abb16f-5c4e-433f-9b0a-d93c1bfb64a0/volumes"
Jul 20 20:27:24.236744 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:24.236708    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"]
Jul 20 20:27:24.237183 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:24.236922    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv" podUID="a93842cd-5ef7-44b2-9e63-e0863e17d4d0" containerName="istio-proxy" containerID="cri-o://4646376a5107061a59a5098e0746f16aae1f86dc1f697f13ee075e1539b9d9d7" gracePeriod=30
Jul 20 20:27:25.061089 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.061055    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"]
Jul 20 20:27:25.061380 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.061365    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59abb16f-5c4e-433f-9b0a-d93c1bfb64a0" containerName="revoke-keys"
Jul 20 20:27:25.061380 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.061376    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="59abb16f-5c4e-433f-9b0a-d93c1bfb64a0" containerName="revoke-keys"
Jul 20 20:27:25.061493 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.061429    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="59abb16f-5c4e-433f-9b0a-d93c1bfb64a0" containerName="revoke-keys"
Jul 20 20:27:25.063221 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.063203    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.067752 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.067729    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-aeaf4876-openshift-default-dockercfg-z2xbm\""
Jul 20 20:27:25.072900 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.072869    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-envoy\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.073022 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.072913    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-data\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.073022 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.072939    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-podinfo\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.073123 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.073022    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab5ab956-a670-408f-970f-ad17ab276c1e-istiod-ca-cert\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.073123 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.073050    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj9vs\" (UniqueName: \"kubernetes.io/projected/ab5ab956-a670-408f-970f-ad17ab276c1e-kube-api-access-xj9vs\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.073230 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.073133    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-workload-socket\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.073230 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.073165    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-token\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.073316 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.073264    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-workload-certs\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.073356 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.073308    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-credential-socket\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.082723 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.082699    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"]
Jul 20 20:27:25.173902 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.173865    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-workload-certs\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.174098 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.173923    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-credential-socket\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.174098 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.173980    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-envoy\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.174098 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.174008    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-data\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.174098 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.174033    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-podinfo\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.174098 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.174074    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab5ab956-a670-408f-970f-ad17ab276c1e-istiod-ca-cert\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.174378 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.174102    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj9vs\" (UniqueName: \"kubernetes.io/projected/ab5ab956-a670-408f-970f-ad17ab276c1e-kube-api-access-xj9vs\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.174378 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.174138    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-workload-socket\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.174378 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.174163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-token\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.174378 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.174348    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-workload-certs\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.174581 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.174410    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-data\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.174733 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.174709    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-workload-socket\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.174950 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.174915    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-credential-socket\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.175116 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.175091    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab5ab956-a670-408f-970f-ad17ab276c1e-istiod-ca-cert\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.176946 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.176922    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-podinfo\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.177056 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.177008    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-envoy\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.183933 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.183897    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-token\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.185015 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.184994    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj9vs\" (UniqueName: \"kubernetes.io/projected/ab5ab956-a670-408f-970f-ad17ab276c1e-kube-api-access-xj9vs\") pod \"e2e-derive-aeaf4876-openshift-default-84478558c-w76z2\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") " pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.373536 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.373439    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:25.517192 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.513575    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"]
Jul 20 20:27:25.518638 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:27:25.518608    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab5ab956_a670_408f_970f_ad17ab276c1e.slice/crio-1b5af55ae9600e39042a0a34c7803b2b475b6bacd3f39b047f8e277b319cf7d4 WatchSource:0}: Error finding container 1b5af55ae9600e39042a0a34c7803b2b475b6bacd3f39b047f8e277b319cf7d4: Status 404 returned error can't find the container with id 1b5af55ae9600e39042a0a34c7803b2b475b6bacd3f39b047f8e277b319cf7d4
Jul 20 20:27:25.520738 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.520701    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:27:25.520831 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.520777    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:27:25.520831 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.520818    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:27:25.693739 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.693644    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2" event={"ID":"ab5ab956-a670-408f-970f-ad17ab276c1e","Type":"ContainerStarted","Data":"d923a1fd4ef3ec2da817d6f4e24532080b4903cb84b4fa79be5d762557d52cc4"}
Jul 20 20:27:25.693739 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.693706    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2" event={"ID":"ab5ab956-a670-408f-970f-ad17ab276c1e","Type":"ContainerStarted","Data":"1b5af55ae9600e39042a0a34c7803b2b475b6bacd3f39b047f8e277b319cf7d4"}
Jul 20 20:27:25.725747 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:25.725678    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2" podStartSLOduration=0.72566282 podStartE2EDuration="725.66282ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:27:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:27:25.724837216 +0000 UTC m=+1860.988907265" watchObservedRunningTime="2026-07-20 20:27:25.72566282 +0000 UTC m=+1860.989732865"
Jul 20 20:27:26.374243 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:26.374202    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:26.378911 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:26.378882    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:26.698231 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:26.698155    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:26.699146 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:26.699128    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:29.481815 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.481790    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:27:29.502794 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.502722    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-envoy\") pod \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") "
Jul 20 20:27:29.502794 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.502759    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-podinfo\") pod \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") "
Jul 20 20:27:29.502794 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.502778    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-data\") pod \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") "
Jul 20 20:27:29.503065 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.502829    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfmjc\" (UniqueName: \"kubernetes.io/projected/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-kube-api-access-nfmjc\") pod \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") "
Jul 20 20:27:29.503065 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.502849    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-credential-socket\") pod \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") "
Jul 20 20:27:29.503065 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.502875    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-workload-socket\") pod \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") "
Jul 20 20:27:29.503065 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.502901    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istiod-ca-cert\") pod \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") "
Jul 20 20:27:29.503065 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.502922    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-workload-certs\") pod \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") "
Jul 20 20:27:29.503065 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.502989    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-token\") pod \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\" (UID: \"a93842cd-5ef7-44b2-9e63-e0863e17d4d0\") "
Jul 20 20:27:29.503366 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.503192    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a93842cd-5ef7-44b2-9e63-e0863e17d4d0" (UID: "a93842cd-5ef7-44b2-9e63-e0863e17d4d0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:27:29.503502 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.503450    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a93842cd-5ef7-44b2-9e63-e0863e17d4d0" (UID: "a93842cd-5ef7-44b2-9e63-e0863e17d4d0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:27:29.503502 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.503466    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a93842cd-5ef7-44b2-9e63-e0863e17d4d0" (UID: "a93842cd-5ef7-44b2-9e63-e0863e17d4d0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:27:29.503615 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.503558    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a93842cd-5ef7-44b2-9e63-e0863e17d4d0" (UID: "a93842cd-5ef7-44b2-9e63-e0863e17d4d0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 20:27:29.504180 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.504154    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-data" (OuterVolumeSpecName: "istio-data") pod "a93842cd-5ef7-44b2-9e63-e0863e17d4d0" (UID: "a93842cd-5ef7-44b2-9e63-e0863e17d4d0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:27:29.506093 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.506070    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a93842cd-5ef7-44b2-9e63-e0863e17d4d0" (UID: "a93842cd-5ef7-44b2-9e63-e0863e17d4d0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:27:29.506257 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.506234    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a93842cd-5ef7-44b2-9e63-e0863e17d4d0" (UID: "a93842cd-5ef7-44b2-9e63-e0863e17d4d0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 20:27:29.506369 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.506349    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-kube-api-access-nfmjc" (OuterVolumeSpecName: "kube-api-access-nfmjc") pod "a93842cd-5ef7-44b2-9e63-e0863e17d4d0" (UID: "a93842cd-5ef7-44b2-9e63-e0863e17d4d0"). InnerVolumeSpecName "kube-api-access-nfmjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:27:29.507118 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.506634    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-token" (OuterVolumeSpecName: "istio-token") pod "a93842cd-5ef7-44b2-9e63-e0863e17d4d0" (UID: "a93842cd-5ef7-44b2-9e63-e0863e17d4d0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:27:29.604215 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.604182    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfmjc\" (UniqueName: \"kubernetes.io/projected/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-kube-api-access-nfmjc\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:27:29.604215 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.604210    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-credential-socket\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:27:29.604215 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.604221    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-workload-socket\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:27:29.604441 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.604233    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istiod-ca-cert\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:27:29.604441 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.604241    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-workload-certs\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:27:29.604441 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.604249    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-token\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:27:29.604441 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.604256    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-envoy\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:27:29.604441 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.604263    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-podinfo\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:27:29.604441 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.604273    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a93842cd-5ef7-44b2-9e63-e0863e17d4d0-istio-data\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:27:29.710794 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.710760    2580 generic.go:356] "Generic (PLEG): container finished" podID="a93842cd-5ef7-44b2-9e63-e0863e17d4d0" containerID="4646376a5107061a59a5098e0746f16aae1f86dc1f697f13ee075e1539b9d9d7" exitCode=0
Jul 20 20:27:29.710975 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.710811    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"
Jul 20 20:27:29.710975 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.710822    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv" event={"ID":"a93842cd-5ef7-44b2-9e63-e0863e17d4d0","Type":"ContainerDied","Data":"4646376a5107061a59a5098e0746f16aae1f86dc1f697f13ee075e1539b9d9d7"}
Jul 20 20:27:29.710975 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.710864    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv" event={"ID":"a93842cd-5ef7-44b2-9e63-e0863e17d4d0","Type":"ContainerDied","Data":"b26b9d252ca1b45477f3d0cf7726e87a4e2d8b116ccc63f7f9c064b6e8724b36"}
Jul 20 20:27:29.710975 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.710884    2580 scope.go:117] "RemoveContainer" containerID="4646376a5107061a59a5098e0746f16aae1f86dc1f697f13ee075e1539b9d9d7"
Jul 20 20:27:29.721573 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.720625    2580 scope.go:117] "RemoveContainer" containerID="145bc6485ea47f58903270510db27d1c2b98170e3ec653227886ed271b4ab59d"
Jul 20 20:27:29.729210 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.729189    2580 scope.go:117] "RemoveContainer" containerID="4646376a5107061a59a5098e0746f16aae1f86dc1f697f13ee075e1539b9d9d7"
Jul 20 20:27:29.729452 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:27:29.729432    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4646376a5107061a59a5098e0746f16aae1f86dc1f697f13ee075e1539b9d9d7\": container with ID starting with 4646376a5107061a59a5098e0746f16aae1f86dc1f697f13ee075e1539b9d9d7 not found: ID does not exist" containerID="4646376a5107061a59a5098e0746f16aae1f86dc1f697f13ee075e1539b9d9d7"
Jul 20 20:27:29.729523 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.729466    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4646376a5107061a59a5098e0746f16aae1f86dc1f697f13ee075e1539b9d9d7"} err="failed to get container status \"4646376a5107061a59a5098e0746f16aae1f86dc1f697f13ee075e1539b9d9d7\": rpc error: code = NotFound desc = could not find container \"4646376a5107061a59a5098e0746f16aae1f86dc1f697f13ee075e1539b9d9d7\": container with ID starting with 4646376a5107061a59a5098e0746f16aae1f86dc1f697f13ee075e1539b9d9d7 not found: ID does not exist"
Jul 20 20:27:29.729523 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.729494    2580 scope.go:117] "RemoveContainer" containerID="145bc6485ea47f58903270510db27d1c2b98170e3ec653227886ed271b4ab59d"
Jul 20 20:27:29.729738 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:27:29.729722    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"145bc6485ea47f58903270510db27d1c2b98170e3ec653227886ed271b4ab59d\": container with ID starting with 145bc6485ea47f58903270510db27d1c2b98170e3ec653227886ed271b4ab59d not found: ID does not exist" containerID="145bc6485ea47f58903270510db27d1c2b98170e3ec653227886ed271b4ab59d"
Jul 20 20:27:29.729800 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.729745    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"145bc6485ea47f58903270510db27d1c2b98170e3ec653227886ed271b4ab59d"} err="failed to get container status \"145bc6485ea47f58903270510db27d1c2b98170e3ec653227886ed271b4ab59d\": rpc error: code = NotFound desc = could not find container \"145bc6485ea47f58903270510db27d1c2b98170e3ec653227886ed271b4ab59d\": container with ID starting with 145bc6485ea47f58903270510db27d1c2b98170e3ec653227886ed271b4ab59d not found: ID does not exist"
Jul 20 20:27:29.740370 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.740350    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"]
Jul 20 20:27:29.752943 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:29.752876    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-47f7982b-openshift-default-c86c48c45-v8jqv"]
Jul 20 20:27:31.175262 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:31.175231    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a93842cd-5ef7-44b2-9e63-e0863e17d4d0" path="/var/lib/kubelet/pods/a93842cd-5ef7-44b2-9e63-e0863e17d4d0/volumes"
Jul 20 20:27:40.758559 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:40.758525    2580 generic.go:356] "Generic (PLEG): container finished" podID="ab5ab956-a670-408f-970f-ad17ab276c1e" containerID="d923a1fd4ef3ec2da817d6f4e24532080b4903cb84b4fa79be5d762557d52cc4" exitCode=0
Jul 20 20:27:40.758998 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:40.758597    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2" event={"ID":"ab5ab956-a670-408f-970f-ad17ab276c1e","Type":"ContainerDied","Data":"d923a1fd4ef3ec2da817d6f4e24532080b4903cb84b4fa79be5d762557d52cc4"}
Jul 20 20:27:40.759047 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:40.759002    2580 scope.go:117] "RemoveContainer" containerID="d923a1fd4ef3ec2da817d6f4e24532080b4903cb84b4fa79be5d762557d52cc4"
Jul 20 20:27:40.759939 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:40.759913    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:27:40.760021 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:40.759987    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:27:40.760021 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:40.760023    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:27:41.698396 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:41.698369    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:41.764129 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:41.764095    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2" event={"ID":"ab5ab956-a670-408f-970f-ad17ab276c1e","Type":"ContainerStarted","Data":"6cd194191a8a20bfc048500ff33ad716ff4dcae88a28248d9e2ffb7caa342ae9"}
Jul 20 20:27:42.373712 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:42.373659    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:42.375238 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:42.375207    2580 patch_prober.go:28] interesting pod/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 20 20:27:42.375460 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:42.375434    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2" podUID="ab5ab956-a670-408f-970f-ad17ab276c1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 20 20:27:43.374985 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:43.374923    2580 patch_prober.go:28] interesting pod/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 20 20:27:43.375624 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:43.375583    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2" podUID="ab5ab956-a670-408f-970f-ad17ab276c1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 20 20:27:44.374287 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:44.374243    2580 patch_prober.go:28] interesting pod/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 20 20:27:44.374507 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:44.374308    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2" podUID="ab5ab956-a670-408f-970f-ad17ab276c1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 20 20:27:45.374710 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:45.374667    2580 patch_prober.go:28] interesting pod/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 20 20:27:45.375335 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:45.375301    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2" podUID="ab5ab956-a670-408f-970f-ad17ab276c1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 20 20:27:46.378972 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:46.378915    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:46.379349 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:46.379206    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:46.379888 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:46.379865    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:27:52.087376 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:52.087282    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-aeaf4876-5n7f5"]
Jul 20 20:27:52.087823 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:52.087731    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a93842cd-5ef7-44b2-9e63-e0863e17d4d0" containerName="istio-proxy"
Jul 20 20:27:52.087823 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:52.087749    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a93842cd-5ef7-44b2-9e63-e0863e17d4d0" containerName="istio-proxy"
Jul 20 20:27:52.087823 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:52.087768    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a93842cd-5ef7-44b2-9e63-e0863e17d4d0" containerName="istio-proxy"
Jul 20 20:27:52.087823 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:52.087777    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a93842cd-5ef7-44b2-9e63-e0863e17d4d0" containerName="istio-proxy"
Jul 20 20:27:52.088052 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:52.087843    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a93842cd-5ef7-44b2-9e63-e0863e17d4d0" containerName="istio-proxy"
Jul 20 20:27:52.088052 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:52.087855    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a93842cd-5ef7-44b2-9e63-e0863e17d4d0" containerName="istio-proxy"
Jul 20 20:27:52.089858 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:52.089838    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-aeaf4876-5n7f5"
Jul 20 20:27:52.093093 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:52.093070    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 20:27:52.093229 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:52.093213    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-628hb\""
Jul 20 20:27:52.101159 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:52.101137    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-aeaf4876-5n7f5"]
Jul 20 20:27:52.213743 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:52.213709    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dda5421-453d-48d2-82d8-bf41727e669e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-aeaf4876-5n7f5\" (UID: \"9dda5421-453d-48d2-82d8-bf41727e669e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-aeaf4876-5n7f5"
Jul 20 20:27:52.314670 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:52.314634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dda5421-453d-48d2-82d8-bf41727e669e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-aeaf4876-5n7f5\" (UID: \"9dda5421-453d-48d2-82d8-bf41727e669e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-aeaf4876-5n7f5"
Jul 20 20:27:52.315373 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:52.315352    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dda5421-453d-48d2-82d8-bf41727e669e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-aeaf4876-5n7f5\" (UID: \"9dda5421-453d-48d2-82d8-bf41727e669e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-aeaf4876-5n7f5"
Jul 20 20:27:52.400377 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:52.400296    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-aeaf4876-5n7f5"
Jul 20 20:27:52.535458 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:52.535353    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-aeaf4876-5n7f5"]
Jul 20 20:27:52.537617 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:27:52.537587    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dda5421_453d_48d2_82d8_bf41727e669e.slice/crio-d5d2bffb3f4423d4e6182ccf6cc788d3801489d658ffc437e5b466f3014e5b27 WatchSource:0}: Error finding container d5d2bffb3f4423d4e6182ccf6cc788d3801489d658ffc437e5b466f3014e5b27: Status 404 returned error can't find the container with id d5d2bffb3f4423d4e6182ccf6cc788d3801489d658ffc437e5b466f3014e5b27
Jul 20 20:27:52.820460 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:52.820427    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-aeaf4876-5n7f5" event={"ID":"9dda5421-453d-48d2-82d8-bf41727e669e","Type":"ContainerStarted","Data":"d5d2bffb3f4423d4e6182ccf6cc788d3801489d658ffc437e5b466f3014e5b27"}
Jul 20 20:27:53.826208 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:53.826176    2580 generic.go:356] "Generic (PLEG): container finished" podID="9dda5421-453d-48d2-82d8-bf41727e669e" containerID="29674872e1df38156b2e6d3667085ea2b53fd46936cab4f9550c71d70511c5fc" exitCode=0
Jul 20 20:27:53.826593 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:53.826228    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-aeaf4876-5n7f5" event={"ID":"9dda5421-453d-48d2-82d8-bf41727e669e","Type":"ContainerDied","Data":"29674872e1df38156b2e6d3667085ea2b53fd46936cab4f9550c71d70511c5fc"}
Jul 20 20:27:54.956589 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:54.956567    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-aeaf4876-5n7f5"
Jul 20 20:27:55.038013 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:55.037976    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dda5421-453d-48d2-82d8-bf41727e669e-maas-api-service-ca\") pod \"9dda5421-453d-48d2-82d8-bf41727e669e\" (UID: \"9dda5421-453d-48d2-82d8-bf41727e669e\") "
Jul 20 20:27:55.038307 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:55.038285    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dda5421-453d-48d2-82d8-bf41727e669e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9dda5421-453d-48d2-82d8-bf41727e669e" (UID: "9dda5421-453d-48d2-82d8-bf41727e669e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 20:27:55.138698 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:55.138620    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dda5421-453d-48d2-82d8-bf41727e669e-maas-api-service-ca\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:27:55.836194 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:55.836167    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-aeaf4876-5n7f5"
Jul 20 20:27:55.836194 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:55.836181    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-aeaf4876-5n7f5" event={"ID":"9dda5421-453d-48d2-82d8-bf41727e669e","Type":"ContainerDied","Data":"d5d2bffb3f4423d4e6182ccf6cc788d3801489d658ffc437e5b466f3014e5b27"}
Jul 20 20:27:55.836391 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:27:55.836224    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5d2bffb3f4423d4e6182ccf6cc788d3801489d658ffc437e5b466f3014e5b27"
Jul 20 20:28:07.200904 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:07.200861    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-aeaf4876-5n7f5"]
Jul 20 20:28:07.202952 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:07.202924    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-aeaf4876-5n7f5"]
Jul 20 20:28:07.364148 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:07.364106    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"]
Jul 20 20:28:07.364374 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:07.364334    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2" podUID="ab5ab956-a670-408f-970f-ad17ab276c1e" containerName="istio-proxy" containerID="cri-o://6cd194191a8a20bfc048500ff33ad716ff4dcae88a28248d9e2ffb7caa342ae9" gracePeriod=30
Jul 20 20:28:09.178693 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:09.178657    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dda5421-453d-48d2-82d8-bf41727e669e" path="/var/lib/kubelet/pods/9dda5421-453d-48d2-82d8-bf41727e669e/volumes"
Jul 20 20:28:12.617450 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.617426    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:28:12.785105 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.785025    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-credential-socket\") pod \"ab5ab956-a670-408f-970f-ad17ab276c1e\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") "
Jul 20 20:28:12.785105 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.785076    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xj9vs\" (UniqueName: \"kubernetes.io/projected/ab5ab956-a670-408f-970f-ad17ab276c1e-kube-api-access-xj9vs\") pod \"ab5ab956-a670-408f-970f-ad17ab276c1e\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") "
Jul 20 20:28:12.785105 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.785101    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab5ab956-a670-408f-970f-ad17ab276c1e-istiod-ca-cert\") pod \"ab5ab956-a670-408f-970f-ad17ab276c1e\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") "
Jul 20 20:28:12.785348 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.785127    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-data\") pod \"ab5ab956-a670-408f-970f-ad17ab276c1e\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") "
Jul 20 20:28:12.785348 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.785255    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-token\") pod \"ab5ab956-a670-408f-970f-ad17ab276c1e\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") "
Jul 20 20:28:12.785348 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.785317    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-envoy\") pod \"ab5ab956-a670-408f-970f-ad17ab276c1e\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") "
Jul 20 20:28:12.785348 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.785316    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ab5ab956-a670-408f-970f-ad17ab276c1e" (UID: "ab5ab956-a670-408f-970f-ad17ab276c1e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:28:12.785558 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.785428    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-workload-socket\") pod \"ab5ab956-a670-408f-970f-ad17ab276c1e\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") "
Jul 20 20:28:12.785558 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.785494    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-workload-certs\") pod \"ab5ab956-a670-408f-970f-ad17ab276c1e\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") "
Jul 20 20:28:12.785558 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.785523    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-podinfo\") pod \"ab5ab956-a670-408f-970f-ad17ab276c1e\" (UID: \"ab5ab956-a670-408f-970f-ad17ab276c1e\") "
Jul 20 20:28:12.785558 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.785534    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab5ab956-a670-408f-970f-ad17ab276c1e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ab5ab956-a670-408f-970f-ad17ab276c1e" (UID: "ab5ab956-a670-408f-970f-ad17ab276c1e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 20:28:12.785749 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.785677    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ab5ab956-a670-408f-970f-ad17ab276c1e" (UID: "ab5ab956-a670-408f-970f-ad17ab276c1e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:28:12.785749 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.785677    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ab5ab956-a670-408f-970f-ad17ab276c1e" (UID: "ab5ab956-a670-408f-970f-ad17ab276c1e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:28:12.785850 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.785799    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-workload-socket\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:28:12.785850 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.785812    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-workload-certs\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:28:12.785850 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.785823    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-credential-socket\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:28:12.785850 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.785836    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab5ab956-a670-408f-970f-ad17ab276c1e-istiod-ca-cert\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:28:12.786150 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.786126    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-data" (OuterVolumeSpecName: "istio-data") pod "ab5ab956-a670-408f-970f-ad17ab276c1e" (UID: "ab5ab956-a670-408f-970f-ad17ab276c1e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:28:12.787377 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.787353    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab5ab956-a670-408f-970f-ad17ab276c1e-kube-api-access-xj9vs" (OuterVolumeSpecName: "kube-api-access-xj9vs") pod "ab5ab956-a670-408f-970f-ad17ab276c1e" (UID: "ab5ab956-a670-408f-970f-ad17ab276c1e"). InnerVolumeSpecName "kube-api-access-xj9vs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:28:12.787644 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.787625    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-token" (OuterVolumeSpecName: "istio-token") pod "ab5ab956-a670-408f-970f-ad17ab276c1e" (UID: "ab5ab956-a670-408f-970f-ad17ab276c1e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:28:12.787806 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.787786    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ab5ab956-a670-408f-970f-ad17ab276c1e" (UID: "ab5ab956-a670-408f-970f-ad17ab276c1e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:28:12.787862 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.787805    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ab5ab956-a670-408f-970f-ad17ab276c1e" (UID: "ab5ab956-a670-408f-970f-ad17ab276c1e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 20:28:12.886299 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.886265    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-podinfo\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:28:12.886299 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.886295    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xj9vs\" (UniqueName: \"kubernetes.io/projected/ab5ab956-a670-408f-970f-ad17ab276c1e-kube-api-access-xj9vs\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:28:12.886299 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.886305    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-data\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:28:12.886528 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.886315    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-token\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:28:12.886528 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.886323    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab5ab956-a670-408f-970f-ad17ab276c1e-istio-envoy\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:28:12.915180 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.915150    2580 generic.go:356] "Generic (PLEG): container finished" podID="ab5ab956-a670-408f-970f-ad17ab276c1e" containerID="6cd194191a8a20bfc048500ff33ad716ff4dcae88a28248d9e2ffb7caa342ae9" exitCode=0
Jul 20 20:28:12.915320 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.915200    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"
Jul 20 20:28:12.915320 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.915198    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2" event={"ID":"ab5ab956-a670-408f-970f-ad17ab276c1e","Type":"ContainerDied","Data":"6cd194191a8a20bfc048500ff33ad716ff4dcae88a28248d9e2ffb7caa342ae9"}
Jul 20 20:28:12.915390 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.915332    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2" event={"ID":"ab5ab956-a670-408f-970f-ad17ab276c1e","Type":"ContainerDied","Data":"1b5af55ae9600e39042a0a34c7803b2b475b6bacd3f39b047f8e277b319cf7d4"}
Jul 20 20:28:12.915390 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.915366    2580 scope.go:117] "RemoveContainer" containerID="6cd194191a8a20bfc048500ff33ad716ff4dcae88a28248d9e2ffb7caa342ae9"
Jul 20 20:28:12.924025 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.924008    2580 scope.go:117] "RemoveContainer" containerID="d923a1fd4ef3ec2da817d6f4e24532080b4903cb84b4fa79be5d762557d52cc4"
Jul 20 20:28:12.932037 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.932021    2580 scope.go:117] "RemoveContainer" containerID="6cd194191a8a20bfc048500ff33ad716ff4dcae88a28248d9e2ffb7caa342ae9"
Jul 20 20:28:12.932626 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:28:12.932608    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cd194191a8a20bfc048500ff33ad716ff4dcae88a28248d9e2ffb7caa342ae9\": container with ID starting with 6cd194191a8a20bfc048500ff33ad716ff4dcae88a28248d9e2ffb7caa342ae9 not found: ID does not exist" containerID="6cd194191a8a20bfc048500ff33ad716ff4dcae88a28248d9e2ffb7caa342ae9"
Jul 20 20:28:12.932678 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.932634    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cd194191a8a20bfc048500ff33ad716ff4dcae88a28248d9e2ffb7caa342ae9"} err="failed to get container status \"6cd194191a8a20bfc048500ff33ad716ff4dcae88a28248d9e2ffb7caa342ae9\": rpc error: code = NotFound desc = could not find container \"6cd194191a8a20bfc048500ff33ad716ff4dcae88a28248d9e2ffb7caa342ae9\": container with ID starting with 6cd194191a8a20bfc048500ff33ad716ff4dcae88a28248d9e2ffb7caa342ae9 not found: ID does not exist"
Jul 20 20:28:12.932678 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.932652    2580 scope.go:117] "RemoveContainer" containerID="d923a1fd4ef3ec2da817d6f4e24532080b4903cb84b4fa79be5d762557d52cc4"
Jul 20 20:28:12.932903 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:28:12.932876    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d923a1fd4ef3ec2da817d6f4e24532080b4903cb84b4fa79be5d762557d52cc4\": container with ID starting with d923a1fd4ef3ec2da817d6f4e24532080b4903cb84b4fa79be5d762557d52cc4 not found: ID does not exist" containerID="d923a1fd4ef3ec2da817d6f4e24532080b4903cb84b4fa79be5d762557d52cc4"
Jul 20 20:28:12.932989 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.932916    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d923a1fd4ef3ec2da817d6f4e24532080b4903cb84b4fa79be5d762557d52cc4"} err="failed to get container status \"d923a1fd4ef3ec2da817d6f4e24532080b4903cb84b4fa79be5d762557d52cc4\": rpc error: code = NotFound desc = could not find container \"d923a1fd4ef3ec2da817d6f4e24532080b4903cb84b4fa79be5d762557d52cc4\": container with ID starting with d923a1fd4ef3ec2da817d6f4e24532080b4903cb84b4fa79be5d762557d52cc4 not found: ID does not exist"
Jul 20 20:28:12.962064 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.962039    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"]
Jul 20 20:28:12.974441 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:12.974419    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-aeaf4876-openshift-default-84478558c-w76z2"]
Jul 20 20:28:13.175442 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:28:13.175404    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab5ab956-a670-408f-970f-ad17ab276c1e" path="/var/lib/kubelet/pods/ab5ab956-a670-408f-970f-ad17ab276c1e/volumes"
Jul 20 20:29:52.356667 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.356628    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"]
Jul 20 20:29:52.357294 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.357052    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab5ab956-a670-408f-970f-ad17ab276c1e" containerName="istio-proxy"
Jul 20 20:29:52.357405 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.357071    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab5ab956-a670-408f-970f-ad17ab276c1e" containerName="istio-proxy"
Jul 20 20:29:52.357525 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.357419    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab5ab956-a670-408f-970f-ad17ab276c1e" containerName="istio-proxy"
Jul 20 20:29:52.357525 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.357435    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab5ab956-a670-408f-970f-ad17ab276c1e" containerName="istio-proxy"
Jul 20 20:29:52.357525 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.357444    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dda5421-453d-48d2-82d8-bf41727e669e" containerName="revoke-keys"
Jul 20 20:29:52.357525 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.357453    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dda5421-453d-48d2-82d8-bf41727e669e" containerName="revoke-keys"
Jul 20 20:29:52.357649 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.357575    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab5ab956-a670-408f-970f-ad17ab276c1e" containerName="istio-proxy"
Jul 20 20:29:52.357649 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.357585    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab5ab956-a670-408f-970f-ad17ab276c1e" containerName="istio-proxy"
Jul 20 20:29:52.357649 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.357592    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dda5421-453d-48d2-82d8-bf41727e669e" containerName="revoke-keys"
Jul 20 20:29:52.360358 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.360343    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.406388 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.406361    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-573f276e-openshift-default-dockercfg-5kg46\""
Jul 20 20:29:52.406684 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.406361    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 20 20:29:52.408415 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.408391    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"]
Jul 20 20:29:52.451082 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.451054    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-data\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.451214 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.451092    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz5zl\" (UniqueName: \"kubernetes.io/projected/44a49fe5-2a44-4bf8-892e-a74726014a22-kube-api-access-lz5zl\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.451214 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.451115    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-workload-socket\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.451214 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.451192    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-token\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.451320 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.451226    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-envoy\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.451320 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.451251    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-credential-socket\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.451320 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.451271    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-workload-certs\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.451320 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.451300    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44a49fe5-2a44-4bf8-892e-a74726014a22-istiod-ca-cert\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.451438 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.451324    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-podinfo\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.551850 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.551820    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz5zl\" (UniqueName: \"kubernetes.io/projected/44a49fe5-2a44-4bf8-892e-a74726014a22-kube-api-access-lz5zl\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.551850 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.551857    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-workload-socket\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.552103 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.551891    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-token\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.552103 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.552009    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-envoy\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.552103 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.552073    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-credential-socket\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.552264 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.552109    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-workload-certs\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.552264 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.552135    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44a49fe5-2a44-4bf8-892e-a74726014a22-istiod-ca-cert\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.552366 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.552275    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-podinfo\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.552366 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.552285    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-workload-socket\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.552366 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.552323    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-data\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.552525 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.552445    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-credential-socket\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.552525 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.552457    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-workload-certs\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.552703 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.552678    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-data\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.552862 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.552841    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44a49fe5-2a44-4bf8-892e-a74726014a22-istiod-ca-cert\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.554167 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.554149    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-envoy\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.554488 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.554472    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-podinfo\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.565830 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.565811    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-token\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.578039 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.578022    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz5zl\" (UniqueName: \"kubernetes.io/projected/44a49fe5-2a44-4bf8-892e-a74726014a22-kube-api-access-lz5zl\") pod \"e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") " pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.675873 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.675791    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:52.831214 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.831172    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"]
Jul 20 20:29:52.834014 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:29:52.833987    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44a49fe5_2a44_4bf8_892e_a74726014a22.slice/crio-cc888b4b3f67bc204b72c2f8fe9f1f5fafd78cc2db751b8f89911bf6dcacc375 WatchSource:0}: Error finding container cc888b4b3f67bc204b72c2f8fe9f1f5fafd78cc2db751b8f89911bf6dcacc375: Status 404 returned error can't find the container with id cc888b4b3f67bc204b72c2f8fe9f1f5fafd78cc2db751b8f89911bf6dcacc375
Jul 20 20:29:52.836101 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.836065    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:29:52.836179 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.836136    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:29:52.836179 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:52.836167    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:29:53.345419 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:53.345379    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6" event={"ID":"44a49fe5-2a44-4bf8-892e-a74726014a22","Type":"ContainerStarted","Data":"f7168850c5a9bec734daa022d86f3d03e656a0f86d273646e1e1b29b9d509054"}
Jul 20 20:29:53.345567 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:53.345444    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6" event={"ID":"44a49fe5-2a44-4bf8-892e-a74726014a22","Type":"ContainerStarted","Data":"cc888b4b3f67bc204b72c2f8fe9f1f5fafd78cc2db751b8f89911bf6dcacc375"}
Jul 20 20:29:53.382328 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:53.382285    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6" podStartSLOduration=1.382269137 podStartE2EDuration="1.382269137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:29:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:29:53.381373621 +0000 UTC m=+2008.645443668" watchObservedRunningTime="2026-07-20 20:29:53.382269137 +0000 UTC m=+2008.646339183"
Jul 20 20:29:53.676607 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:53.676523    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:53.681334 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:53.681308    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:54.349423 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:54.349394    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:29:54.350190 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:29:54.350172    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:30:00.158629 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:00.158595    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742990-zqbmp"]
Jul 20 20:30:00.161701 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:00.161682    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742990-zqbmp"
Jul 20 20:30:00.164928 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:00.164908    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-628hb\""
Jul 20 20:30:00.171060 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:00.171045    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742990-zqbmp"
Jul 20 20:30:00.175469 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:00.175445    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-573f276e-29742990-drcx9"]
Jul 20 20:30:00.178891 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:00.178868    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742990-zqbmp"]
Jul 20 20:30:00.179023 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:00.178989    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-573f276e-29742990-drcx9"
Jul 20 20:30:00.199545 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:00.199520    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-573f276e-29742990-drcx9"]
Jul 20 20:30:00.204643 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:00.204624    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-573f276e-29742990-drcx9"
Jul 20 20:30:00.328526 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:00.328498    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742990-zqbmp"]
Jul 20 20:30:00.331164 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:30:00.331139    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0d3ba47_4119_4152_9aad_8f7780629cc6.slice/crio-951058a899b03243ff31271597306e4abada22f0bd73b78d65f43e74d2069409 WatchSource:0}: Error finding container 951058a899b03243ff31271597306e4abada22f0bd73b78d65f43e74d2069409: Status 404 returned error can't find the container with id 951058a899b03243ff31271597306e4abada22f0bd73b78d65f43e74d2069409
Jul 20 20:30:00.362380 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:00.362358    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-573f276e-29742990-drcx9"]
Jul 20 20:30:00.364832 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:30:00.364809    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b4d6e71_457a_4021_9b96_777fa3eadf90.slice/crio-6ea69bde12d2a1f0992ee9bcbf3bafad436a1b5019d9f4bb8d9e70b17a1bb378 WatchSource:0}: Error finding container 6ea69bde12d2a1f0992ee9bcbf3bafad436a1b5019d9f4bb8d9e70b17a1bb378: Status 404 returned error can't find the container with id 6ea69bde12d2a1f0992ee9bcbf3bafad436a1b5019d9f4bb8d9e70b17a1bb378
Jul 20 20:30:00.382305 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:00.382270    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-573f276e-29742990-drcx9" event={"ID":"8b4d6e71-457a-4021-9b96-777fa3eadf90","Type":"ContainerStarted","Data":"6ea69bde12d2a1f0992ee9bcbf3bafad436a1b5019d9f4bb8d9e70b17a1bb378"}
Jul 20 20:30:00.383482 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:00.383460    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742990-zqbmp" event={"ID":"f0d3ba47-4119-4152-9aad-8f7780629cc6","Type":"ContainerStarted","Data":"951058a899b03243ff31271597306e4abada22f0bd73b78d65f43e74d2069409"}
Jul 20 20:30:01.390602 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:01.390576    2580 generic.go:356] "Generic (PLEG): container finished" podID="8b4d6e71-457a-4021-9b96-777fa3eadf90" containerID="84306ced7f71da3160b098ecd14b5dd593921e2fed1fd6625078c9414a906460" exitCode=7
Jul 20 20:30:01.390974 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:01.390665    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-573f276e-29742990-drcx9" event={"ID":"8b4d6e71-457a-4021-9b96-777fa3eadf90","Type":"ContainerDied","Data":"84306ced7f71da3160b098ecd14b5dd593921e2fed1fd6625078c9414a906460"}
Jul 20 20:30:01.390974 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:01.390827    2580 scope.go:117] "RemoveContainer" containerID="84306ced7f71da3160b098ecd14b5dd593921e2fed1fd6625078c9414a906460"
Jul 20 20:30:01.392169 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:01.392147    2580 generic.go:356] "Generic (PLEG): container finished" podID="f0d3ba47-4119-4152-9aad-8f7780629cc6" containerID="53ce205711de8b44f6e080de79ac912a903dbbb8c0ff083506c9f27fb90e3e8b" exitCode=0
Jul 20 20:30:01.392282 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:01.392218    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742990-zqbmp" event={"ID":"f0d3ba47-4119-4152-9aad-8f7780629cc6","Type":"ContainerDied","Data":"53ce205711de8b44f6e080de79ac912a903dbbb8c0ff083506c9f27fb90e3e8b"}
Jul 20 20:30:02.398630 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:02.398531    2580 generic.go:356] "Generic (PLEG): container finished" podID="8b4d6e71-457a-4021-9b96-777fa3eadf90" containerID="f9130c3fb9a41174db683fce22987fb73495d35e2a6e5fbfecd2232411d2919c" exitCode=7
Jul 20 20:30:02.398630 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:02.398579    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-573f276e-29742990-drcx9" event={"ID":"8b4d6e71-457a-4021-9b96-777fa3eadf90","Type":"ContainerDied","Data":"f9130c3fb9a41174db683fce22987fb73495d35e2a6e5fbfecd2232411d2919c"}
Jul 20 20:30:02.399183 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:02.398645    2580 scope.go:117] "RemoveContainer" containerID="84306ced7f71da3160b098ecd14b5dd593921e2fed1fd6625078c9414a906460"
Jul 20 20:30:02.399183 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:02.398783    2580 scope.go:117] "RemoveContainer" containerID="f9130c3fb9a41174db683fce22987fb73495d35e2a6e5fbfecd2232411d2919c"
Jul 20 20:30:02.399183 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:30:02.399048    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-mt-573f276e-29742990-drcx9_odh-ai-gateway-infra(8b4d6e71-457a-4021-9b96-777fa3eadf90)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-573f276e-29742990-drcx9" podUID="8b4d6e71-457a-4021-9b96-777fa3eadf90"
Jul 20 20:30:02.534447 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:02.534422    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742990-zqbmp"
Jul 20 20:30:03.404187 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:03.404157    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742990-zqbmp"
Jul 20 20:30:03.404605 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:03.404196    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742990-zqbmp" event={"ID":"f0d3ba47-4119-4152-9aad-8f7780629cc6","Type":"ContainerDied","Data":"951058a899b03243ff31271597306e4abada22f0bd73b78d65f43e74d2069409"}
Jul 20 20:30:03.404605 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:03.404260    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="951058a899b03243ff31271597306e4abada22f0bd73b78d65f43e74d2069409"
Jul 20 20:30:03.406025 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:03.406001    2580 scope.go:117] "RemoveContainer" containerID="f9130c3fb9a41174db683fce22987fb73495d35e2a6e5fbfecd2232411d2919c"
Jul 20 20:30:03.406343 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:30:03.406246    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-mt-573f276e-29742990-drcx9_odh-ai-gateway-infra(8b4d6e71-457a-4021-9b96-777fa3eadf90)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-573f276e-29742990-drcx9" podUID="8b4d6e71-457a-4021-9b96-777fa3eadf90"
Jul 20 20:30:08.358677 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:08.358640    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-573f276e-kdpv6"]
Jul 20 20:30:08.359237 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:08.359070    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0d3ba47-4119-4152-9aad-8f7780629cc6" containerName="cleanup"
Jul 20 20:30:08.359237 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:08.359084    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0d3ba47-4119-4152-9aad-8f7780629cc6" containerName="cleanup"
Jul 20 20:30:08.359237 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:08.359182    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0d3ba47-4119-4152-9aad-8f7780629cc6" containerName="cleanup"
Jul 20 20:30:08.362479 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:08.362455    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-573f276e-kdpv6"
Jul 20 20:30:08.366549 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:08.366517    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 20:30:08.406283 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:08.406243    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-573f276e-kdpv6"]
Jul 20 20:30:08.491134 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:08.491092    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0089e80b-4b7f-4f41-a671-d863ce7ca4d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-573f276e-kdpv6\" (UID: \"0089e80b-4b7f-4f41-a671-d863ce7ca4d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-573f276e-kdpv6"
Jul 20 20:30:08.591973 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:08.591938    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0089e80b-4b7f-4f41-a671-d863ce7ca4d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-573f276e-kdpv6\" (UID: \"0089e80b-4b7f-4f41-a671-d863ce7ca4d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-573f276e-kdpv6"
Jul 20 20:30:08.592570 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:08.592549    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0089e80b-4b7f-4f41-a671-d863ce7ca4d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-573f276e-kdpv6\" (UID: \"0089e80b-4b7f-4f41-a671-d863ce7ca4d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-573f276e-kdpv6"
Jul 20 20:30:08.675341 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:08.675263    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-573f276e-kdpv6"
Jul 20 20:30:08.843312 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:08.843278    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-573f276e-kdpv6"]
Jul 20 20:30:08.844639 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:30:08.844611    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0089e80b_4b7f_4f41_a671_d863ce7ca4d7.slice/crio-d5657198c9f45b94fe2a3eb5b827b44dd8db6121b0be71063d00282a90b7578c WatchSource:0}: Error finding container d5657198c9f45b94fe2a3eb5b827b44dd8db6121b0be71063d00282a90b7578c: Status 404 returned error can't find the container with id d5657198c9f45b94fe2a3eb5b827b44dd8db6121b0be71063d00282a90b7578c
Jul 20 20:30:09.434246 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:09.434156    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-573f276e-kdpv6" event={"ID":"0089e80b-4b7f-4f41-a671-d863ce7ca4d7","Type":"ContainerStarted","Data":"452e01e8115799fedf40b333e25aa2f6456dc1f6cce45b5ed6633ad25239e9b1"}
Jul 20 20:30:09.434246 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:09.434224    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-573f276e-kdpv6" event={"ID":"0089e80b-4b7f-4f41-a671-d863ce7ca4d7","Type":"ContainerStarted","Data":"d5657198c9f45b94fe2a3eb5b827b44dd8db6121b0be71063d00282a90b7578c"}
Jul 20 20:30:09.460075 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:09.460010    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-573f276e-kdpv6" podStartSLOduration=1.459987038 podStartE2EDuration="1.459987038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:30:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:30:09.455694698 +0000 UTC m=+2024.719764743" watchObservedRunningTime="2026-07-20 20:30:09.459987038 +0000 UTC m=+2024.724057090"
Jul 20 20:30:09.908019 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:09.906633    2580 patch_prober.go:28] interesting pod/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": read tcp 10.133.0.2:48308->10.133.0.38:15021: read: connection reset by peer" start-of-body=
Jul 20 20:30:09.908019 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:09.906691    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6" podUID="44a49fe5-2a44-4bf8-892e-a74726014a22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": read tcp 10.133.0.2:48308->10.133.0.38:15021: read: connection reset by peer"
Jul 20 20:30:10.440059 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:10.440023    2580 generic.go:356] "Generic (PLEG): container finished" podID="0089e80b-4b7f-4f41-a671-d863ce7ca4d7" containerID="452e01e8115799fedf40b333e25aa2f6456dc1f6cce45b5ed6633ad25239e9b1" exitCode=0
Jul 20 20:30:10.440510 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:10.440104    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-573f276e-kdpv6" event={"ID":"0089e80b-4b7f-4f41-a671-d863ce7ca4d7","Type":"ContainerDied","Data":"452e01e8115799fedf40b333e25aa2f6456dc1f6cce45b5ed6633ad25239e9b1"}
Jul 20 20:30:10.441634 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:10.441603    2580 generic.go:356] "Generic (PLEG): container finished" podID="44a49fe5-2a44-4bf8-892e-a74726014a22" containerID="f7168850c5a9bec734daa022d86f3d03e656a0f86d273646e1e1b29b9d509054" exitCode=0
Jul 20 20:30:10.441738 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:10.441675    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6" event={"ID":"44a49fe5-2a44-4bf8-892e-a74726014a22","Type":"ContainerDied","Data":"f7168850c5a9bec734daa022d86f3d03e656a0f86d273646e1e1b29b9d509054"}
Jul 20 20:30:10.442040 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:10.442026    2580 scope.go:117] "RemoveContainer" containerID="f7168850c5a9bec734daa022d86f3d03e656a0f86d273646e1e1b29b9d509054"
Jul 20 20:30:10.442920 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:10.442896    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:30:10.442997 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:10.442986    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:30:10.443031 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:10.443022    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:30:11.449182 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:11.449146    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6" event={"ID":"44a49fe5-2a44-4bf8-892e-a74726014a22","Type":"ContainerStarted","Data":"3306400869b1b25d8ecc8b8b69de33c8c650c8514ff559efe155f827df99e439"}
Jul 20 20:30:11.630214 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:11.630189    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-573f276e-kdpv6"
Jul 20 20:30:11.678445 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:11.678398    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:30:11.678650 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:11.678535    2580 patch_prober.go:28] interesting pod/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 20 20:30:11.678650 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:11.678577    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6" podUID="44a49fe5-2a44-4bf8-892e-a74726014a22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 20 20:30:11.720044 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:11.719931    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0089e80b-4b7f-4f41-a671-d863ce7ca4d7-maas-api-service-ca\") pod \"0089e80b-4b7f-4f41-a671-d863ce7ca4d7\" (UID: \"0089e80b-4b7f-4f41-a671-d863ce7ca4d7\") "
Jul 20 20:30:11.720267 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:11.720245    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0089e80b-4b7f-4f41-a671-d863ce7ca4d7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0089e80b-4b7f-4f41-a671-d863ce7ca4d7" (UID: "0089e80b-4b7f-4f41-a671-d863ce7ca4d7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 20:30:11.820867 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:11.820836    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0089e80b-4b7f-4f41-a671-d863ce7ca4d7-maas-api-service-ca\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:30:12.454593 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:12.454554    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-573f276e-kdpv6" event={"ID":"0089e80b-4b7f-4f41-a671-d863ce7ca4d7","Type":"ContainerDied","Data":"d5657198c9f45b94fe2a3eb5b827b44dd8db6121b0be71063d00282a90b7578c"}
Jul 20 20:30:12.454593 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:12.454594    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-573f276e-kdpv6"
Jul 20 20:30:12.455078 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:12.454618    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5657198c9f45b94fe2a3eb5b827b44dd8db6121b0be71063d00282a90b7578c"
Jul 20 20:30:12.677652 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:12.677207    2580 patch_prober.go:28] interesting pod/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 20 20:30:12.677652 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:12.677283    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6" podUID="44a49fe5-2a44-4bf8-892e-a74726014a22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 20 20:30:13.676634 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:13.676586    2580 patch_prober.go:28] interesting pod/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 20 20:30:13.677142 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:13.676656    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6" podUID="44a49fe5-2a44-4bf8-892e-a74726014a22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 20 20:30:14.170455 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:14.170419    2580 scope.go:117] "RemoveContainer" containerID="f9130c3fb9a41174db683fce22987fb73495d35e2a6e5fbfecd2232411d2919c"
Jul 20 20:30:14.677066 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:14.677024    2580 patch_prober.go:28] interesting pod/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 20 20:30:14.677548 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:14.677097    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6" podUID="44a49fe5-2a44-4bf8-892e-a74726014a22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 20 20:30:15.471253 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:15.471211    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-573f276e-29742990-drcx9" event={"ID":"8b4d6e71-457a-4021-9b96-777fa3eadf90","Type":"ContainerStarted","Data":"76c4fa1d9ee1f8d1249450cc31ec43de570da20c96f8c92aa5a89f984e25cf74"}
Jul 20 20:30:15.491054 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:15.490987    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-573f276e-29742990-drcx9" podStartSLOduration=15.490951149 podStartE2EDuration="15.490951149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:30:15.489822773 +0000 UTC m=+2030.753892822" watchObservedRunningTime="2026-07-20 20:30:15.490951149 +0000 UTC m=+2030.755021194"
Jul 20 20:30:15.679743 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:15.679713    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:30:15.680142 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:15.680085    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:30:15.680745 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:15.680729    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:30:16.477182 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:16.477151    2580 generic.go:356] "Generic (PLEG): container finished" podID="8b4d6e71-457a-4021-9b96-777fa3eadf90" containerID="76c4fa1d9ee1f8d1249450cc31ec43de570da20c96f8c92aa5a89f984e25cf74" exitCode=0
Jul 20 20:30:16.477353 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:16.477220    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-573f276e-29742990-drcx9" event={"ID":"8b4d6e71-457a-4021-9b96-777fa3eadf90","Type":"ContainerDied","Data":"76c4fa1d9ee1f8d1249450cc31ec43de570da20c96f8c92aa5a89f984e25cf74"}
Jul 20 20:30:16.477353 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:16.477280    2580 scope.go:117] "RemoveContainer" containerID="f9130c3fb9a41174db683fce22987fb73495d35e2a6e5fbfecd2232411d2919c"
Jul 20 20:30:16.501564 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:16.501537    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-573f276e-29742990-drcx9"]
Jul 20 20:30:17.612992 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:17.612943    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-573f276e-29742990-drcx9"
Jul 20 20:30:18.494041 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:18.494008    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-573f276e-29742990-drcx9"
Jul 20 20:30:18.494041 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:18.494024    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-573f276e-29742990-drcx9" event={"ID":"8b4d6e71-457a-4021-9b96-777fa3eadf90","Type":"ContainerDied","Data":"6ea69bde12d2a1f0992ee9bcbf3bafad436a1b5019d9f4bb8d9e70b17a1bb378"}
Jul 20 20:30:18.494235 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:18.494103    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ea69bde12d2a1f0992ee9bcbf3bafad436a1b5019d9f4bb8d9e70b17a1bb378"
Jul 20 20:30:18.526793 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:18.526762    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-573f276e-29742990-drcx9"]
Jul 20 20:30:18.532658 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:18.532637    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-573f276e-29742990-drcx9"]
Jul 20 20:30:19.175175 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:19.175144    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b4d6e71-457a-4021-9b96-777fa3eadf90" path="/var/lib/kubelet/pods/8b4d6e71-457a-4021-9b96-777fa3eadf90/volumes"
Jul 20 20:30:23.399109 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:23.399072    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-573f276e-kdpv6"]
Jul 20 20:30:23.402216 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:23.402189    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-573f276e-kdpv6"]
Jul 20 20:30:25.177755 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:25.177718    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0089e80b-4b7f-4f41-a671-d863ce7ca4d7" path="/var/lib/kubelet/pods/0089e80b-4b7f-4f41-a671-d863ce7ca4d7/volumes"
Jul 20 20:30:30.565289 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:30.565250    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"]
Jul 20 20:30:30.565753 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:30.565470    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6" podUID="44a49fe5-2a44-4bf8-892e-a74726014a22" containerName="istio-proxy" containerID="cri-o://3306400869b1b25d8ecc8b8b69de33c8c650c8514ff559efe155f827df99e439" gracePeriod=30
Jul 20 20:30:30.682155 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:30.682120    2580 patch_prober.go:28] interesting pod/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 20 20:30:30.682321 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:30.682185    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6" podUID="44a49fe5-2a44-4bf8-892e-a74726014a22" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 20 20:30:35.818361 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:35.818338    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:30:35.923421 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:35.923332    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-envoy\") pod \"44a49fe5-2a44-4bf8-892e-a74726014a22\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") "
Jul 20 20:30:35.923421 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:35.923381    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-workload-certs\") pod \"44a49fe5-2a44-4bf8-892e-a74726014a22\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") "
Jul 20 20:30:35.923421 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:35.923413    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44a49fe5-2a44-4bf8-892e-a74726014a22-istiod-ca-cert\") pod \"44a49fe5-2a44-4bf8-892e-a74726014a22\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") "
Jul 20 20:30:35.923683 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:35.923433    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-workload-socket\") pod \"44a49fe5-2a44-4bf8-892e-a74726014a22\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") "
Jul 20 20:30:35.923683 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:35.923461    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lz5zl\" (UniqueName: \"kubernetes.io/projected/44a49fe5-2a44-4bf8-892e-a74726014a22-kube-api-access-lz5zl\") pod \"44a49fe5-2a44-4bf8-892e-a74726014a22\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") "
Jul 20 20:30:35.923683 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:35.923479    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-podinfo\") pod \"44a49fe5-2a44-4bf8-892e-a74726014a22\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") "
Jul 20 20:30:35.923683 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:35.923496    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-data\") pod \"44a49fe5-2a44-4bf8-892e-a74726014a22\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") "
Jul 20 20:30:35.923683 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:35.923518    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-credential-socket\") pod \"44a49fe5-2a44-4bf8-892e-a74726014a22\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") "
Jul 20 20:30:35.923683 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:35.923563    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-token\") pod \"44a49fe5-2a44-4bf8-892e-a74726014a22\" (UID: \"44a49fe5-2a44-4bf8-892e-a74726014a22\") "
Jul 20 20:30:35.924008 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:35.923692    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "44a49fe5-2a44-4bf8-892e-a74726014a22" (UID: "44a49fe5-2a44-4bf8-892e-a74726014a22"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:30:35.924008 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:35.923825    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "44a49fe5-2a44-4bf8-892e-a74726014a22" (UID: "44a49fe5-2a44-4bf8-892e-a74726014a22"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:30:35.924008 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:35.923847    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-workload-certs\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:30:35.924008 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:35.923890    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44a49fe5-2a44-4bf8-892e-a74726014a22-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "44a49fe5-2a44-4bf8-892e-a74726014a22" (UID: "44a49fe5-2a44-4bf8-892e-a74726014a22"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 20:30:35.924189 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:35.924045    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "44a49fe5-2a44-4bf8-892e-a74726014a22" (UID: "44a49fe5-2a44-4bf8-892e-a74726014a22"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:30:35.924463 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:35.924436    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-data" (OuterVolumeSpecName: "istio-data") pod "44a49fe5-2a44-4bf8-892e-a74726014a22" (UID: "44a49fe5-2a44-4bf8-892e-a74726014a22"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:30:35.926265 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:35.926225    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "44a49fe5-2a44-4bf8-892e-a74726014a22" (UID: "44a49fe5-2a44-4bf8-892e-a74726014a22"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:30:35.926265 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:35.926232    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-token" (OuterVolumeSpecName: "istio-token") pod "44a49fe5-2a44-4bf8-892e-a74726014a22" (UID: "44a49fe5-2a44-4bf8-892e-a74726014a22"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:30:35.926404 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:35.926315    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44a49fe5-2a44-4bf8-892e-a74726014a22-kube-api-access-lz5zl" (OuterVolumeSpecName: "kube-api-access-lz5zl") pod "44a49fe5-2a44-4bf8-892e-a74726014a22" (UID: "44a49fe5-2a44-4bf8-892e-a74726014a22"). InnerVolumeSpecName "kube-api-access-lz5zl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:30:35.926404 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:35.926319    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "44a49fe5-2a44-4bf8-892e-a74726014a22" (UID: "44a49fe5-2a44-4bf8-892e-a74726014a22"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 20:30:36.024744 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:36.024695    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44a49fe5-2a44-4bf8-892e-a74726014a22-istiod-ca-cert\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:30:36.024744 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:36.024735    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-workload-socket\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:30:36.024744 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:36.024746    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lz5zl\" (UniqueName: \"kubernetes.io/projected/44a49fe5-2a44-4bf8-892e-a74726014a22-kube-api-access-lz5zl\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:30:36.024744 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:36.024756    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-podinfo\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:30:36.025095 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:36.024766    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-data\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:30:36.025095 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:36.024774    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-credential-socket\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:30:36.025095 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:36.024781    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-token\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:30:36.025095 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:36.024789    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44a49fe5-2a44-4bf8-892e-a74726014a22-istio-envoy\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:30:36.573991 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:36.573939    2580 generic.go:356] "Generic (PLEG): container finished" podID="44a49fe5-2a44-4bf8-892e-a74726014a22" containerID="3306400869b1b25d8ecc8b8b69de33c8c650c8514ff559efe155f827df99e439" exitCode=0
Jul 20 20:30:36.574163 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:36.574029    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"
Jul 20 20:30:36.574163 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:36.574028    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6" event={"ID":"44a49fe5-2a44-4bf8-892e-a74726014a22","Type":"ContainerDied","Data":"3306400869b1b25d8ecc8b8b69de33c8c650c8514ff559efe155f827df99e439"}
Jul 20 20:30:36.574163 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:36.574093    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6" event={"ID":"44a49fe5-2a44-4bf8-892e-a74726014a22","Type":"ContainerDied","Data":"cc888b4b3f67bc204b72c2f8fe9f1f5fafd78cc2db751b8f89911bf6dcacc375"}
Jul 20 20:30:36.574163 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:36.574123    2580 scope.go:117] "RemoveContainer" containerID="3306400869b1b25d8ecc8b8b69de33c8c650c8514ff559efe155f827df99e439"
Jul 20 20:30:36.583452 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:36.583437    2580 scope.go:117] "RemoveContainer" containerID="f7168850c5a9bec734daa022d86f3d03e656a0f86d273646e1e1b29b9d509054"
Jul 20 20:30:36.591018 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:36.590997    2580 scope.go:117] "RemoveContainer" containerID="3306400869b1b25d8ecc8b8b69de33c8c650c8514ff559efe155f827df99e439"
Jul 20 20:30:36.591274 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:30:36.591254    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3306400869b1b25d8ecc8b8b69de33c8c650c8514ff559efe155f827df99e439\": container with ID starting with 3306400869b1b25d8ecc8b8b69de33c8c650c8514ff559efe155f827df99e439 not found: ID does not exist" containerID="3306400869b1b25d8ecc8b8b69de33c8c650c8514ff559efe155f827df99e439"
Jul 20 20:30:36.591351 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:36.591283    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3306400869b1b25d8ecc8b8b69de33c8c650c8514ff559efe155f827df99e439"} err="failed to get container status \"3306400869b1b25d8ecc8b8b69de33c8c650c8514ff559efe155f827df99e439\": rpc error: code = NotFound desc = could not find container \"3306400869b1b25d8ecc8b8b69de33c8c650c8514ff559efe155f827df99e439\": container with ID starting with 3306400869b1b25d8ecc8b8b69de33c8c650c8514ff559efe155f827df99e439 not found: ID does not exist"
Jul 20 20:30:36.591351 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:36.591302    2580 scope.go:117] "RemoveContainer" containerID="f7168850c5a9bec734daa022d86f3d03e656a0f86d273646e1e1b29b9d509054"
Jul 20 20:30:36.591551 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:30:36.591536    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7168850c5a9bec734daa022d86f3d03e656a0f86d273646e1e1b29b9d509054\": container with ID starting with f7168850c5a9bec734daa022d86f3d03e656a0f86d273646e1e1b29b9d509054 not found: ID does not exist" containerID="f7168850c5a9bec734daa022d86f3d03e656a0f86d273646e1e1b29b9d509054"
Jul 20 20:30:36.591600 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:36.591555    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7168850c5a9bec734daa022d86f3d03e656a0f86d273646e1e1b29b9d509054"} err="failed to get container status \"f7168850c5a9bec734daa022d86f3d03e656a0f86d273646e1e1b29b9d509054\": rpc error: code = NotFound desc = could not find container \"f7168850c5a9bec734daa022d86f3d03e656a0f86d273646e1e1b29b9d509054\": container with ID starting with f7168850c5a9bec734daa022d86f3d03e656a0f86d273646e1e1b29b9d509054 not found: ID does not exist"
Jul 20 20:30:36.597617 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:36.597596    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"]
Jul 20 20:30:36.601892 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:36.601870    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-573f276e-openshift-default-68bf8f8f8c-nnwg6"]
Jul 20 20:30:37.176132 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:37.176103    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44a49fe5-2a44-4bf8-892e-a74726014a22" path="/var/lib/kubelet/pods/44a49fe5-2a44-4bf8-892e-a74726014a22/volumes"
Jul 20 20:30:52.005784 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.005705    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"]
Jul 20 20:30:52.006214 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.006053    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44a49fe5-2a44-4bf8-892e-a74726014a22" containerName="istio-proxy"
Jul 20 20:30:52.006214 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.006069    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="44a49fe5-2a44-4bf8-892e-a74726014a22" containerName="istio-proxy"
Jul 20 20:30:52.006214 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.006080    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b4d6e71-457a-4021-9b96-777fa3eadf90" containerName="cleanup"
Jul 20 20:30:52.006214 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.006085    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b4d6e71-457a-4021-9b96-777fa3eadf90" containerName="cleanup"
Jul 20 20:30:52.006214 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.006098    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44a49fe5-2a44-4bf8-892e-a74726014a22" containerName="istio-proxy"
Jul 20 20:30:52.006214 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.006104    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="44a49fe5-2a44-4bf8-892e-a74726014a22" containerName="istio-proxy"
Jul 20 20:30:52.006214 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.006110    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b4d6e71-457a-4021-9b96-777fa3eadf90" containerName="cleanup"
Jul 20 20:30:52.006214 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.006115    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b4d6e71-457a-4021-9b96-777fa3eadf90" containerName="cleanup"
Jul 20 20:30:52.006214 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.006122    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0089e80b-4b7f-4f41-a671-d863ce7ca4d7" containerName="revoke-keys"
Jul 20 20:30:52.006214 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.006130    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0089e80b-4b7f-4f41-a671-d863ce7ca4d7" containerName="revoke-keys"
Jul 20 20:30:52.006214 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.006140    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b4d6e71-457a-4021-9b96-777fa3eadf90" containerName="cleanup"
Jul 20 20:30:52.006214 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.006145    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b4d6e71-457a-4021-9b96-777fa3eadf90" containerName="cleanup"
Jul 20 20:30:52.006214 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.006195    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="44a49fe5-2a44-4bf8-892e-a74726014a22" containerName="istio-proxy"
Jul 20 20:30:52.006214 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.006209    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b4d6e71-457a-4021-9b96-777fa3eadf90" containerName="cleanup"
Jul 20 20:30:52.006214 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.006218    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b4d6e71-457a-4021-9b96-777fa3eadf90" containerName="cleanup"
Jul 20 20:30:52.006652 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.006224    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0089e80b-4b7f-4f41-a671-d863ce7ca4d7" containerName="revoke-keys"
Jul 20 20:30:52.006652 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.006230    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b4d6e71-457a-4021-9b96-777fa3eadf90" containerName="cleanup"
Jul 20 20:30:52.010433 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.010415    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.013655 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.013634    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-fa45191d-openshift-default-dockercfg-4tjm6\""
Jul 20 20:30:52.014014 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.013997    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 20 20:30:52.029531 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.029502    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"]
Jul 20 20:30:52.052403 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.052378    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-data\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.052532 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.052412    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-envoy\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.052532 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.052452    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nklzp\" (UniqueName: \"kubernetes.io/projected/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-kube-api-access-nklzp\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.052608 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.052532    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-podinfo\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.052608 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.052575    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-workload-certs\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.052673 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.052614    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-token\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.052673 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.052647    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-workload-socket\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.052737 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.052671    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-credential-socket\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.052737 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.052695    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istiod-ca-cert\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.153832 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.153790    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-podinfo\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.153832 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.153839    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-workload-certs\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.154116 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.153879    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-token\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.154116 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.153915    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-workload-socket\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.154116 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.153940    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-credential-socket\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.154116 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.153987    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istiod-ca-cert\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.154116 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.154030    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-data\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.154116 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.154061    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-envoy\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.154417 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.154128    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nklzp\" (UniqueName: \"kubernetes.io/projected/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-kube-api-access-nklzp\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.154417 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.154385    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-workload-socket\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.154555 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.154533    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-data\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.154616 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.154568    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-credential-socket\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.154669 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.154655    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-workload-certs\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.155014 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.154992    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istiod-ca-cert\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.156465 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.156440    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-envoy\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.156571 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.156553    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-podinfo\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.165593 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.165568    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-token\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.165713 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.165579    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nklzp\" (UniqueName: \"kubernetes.io/projected/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-kube-api-access-nklzp\") pod \"e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") " pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.320074 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.320035    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:52.463360 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.463323    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"]
Jul 20 20:30:52.466379 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:30:52.466352    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bc8e78f_68c4_4bbc_b9a7_34c956ac7ab0.slice/crio-25c12324582d889593399a10ecf4c100e0af881c86d8827eb7e6f2b77c1745a6 WatchSource:0}: Error finding container 25c12324582d889593399a10ecf4c100e0af881c86d8827eb7e6f2b77c1745a6: Status 404 returned error can't find the container with id 25c12324582d889593399a10ecf4c100e0af881c86d8827eb7e6f2b77c1745a6
Jul 20 20:30:52.468498 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.468460    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:30:52.468569 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.468533    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:30:52.468569 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.468563    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:30:52.648297 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.648199    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9" event={"ID":"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0","Type":"ContainerStarted","Data":"392860fb648f0faa590dce13fd620f4d47b6b95ce72b0d65966cb2f24de75917"}
Jul 20 20:30:52.648297 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.648270    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9" event={"ID":"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0","Type":"ContainerStarted","Data":"25c12324582d889593399a10ecf4c100e0af881c86d8827eb7e6f2b77c1745a6"}
Jul 20 20:30:52.676675 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:52.676617    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9" podStartSLOduration=1.676597053 podStartE2EDuration="1.676597053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:30:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:30:52.67538976 +0000 UTC m=+2067.939459806" watchObservedRunningTime="2026-07-20 20:30:52.676597053 +0000 UTC m=+2067.940667098"
Jul 20 20:30:53.320953 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:53.320916    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:53.325818 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:53.325792    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:53.653142 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:53.653050    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:30:53.654205 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:30:53.654187    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:31:03.658493 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.658456    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"]
Jul 20 20:31:03.658877 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.658797    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="44a49fe5-2a44-4bf8-892e-a74726014a22" containerName="istio-proxy"
Jul 20 20:31:03.661493 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.661478    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:03.665418 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.665185    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-fa45191d\"/\"kube-root-ca.crt\""
Jul 20 20:31:03.667441 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.667415    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-fa45191d\"/\"e2e-lifecycle-model-fa45191d-kserve-self-signed-certs\""
Jul 20 20:31:03.667828 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.667521    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-fa45191d\"/\"default-dockercfg-c5f92\""
Jul 20 20:31:03.668100 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.667621    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-fa45191d\"/\"openshift-service-ca.crt\""
Jul 20 20:31:03.676774 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.676731    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"]
Jul 20 20:31:03.758997 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.758946    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qq7z\" (UniqueName: \"kubernetes.io/projected/0918ab9b-f101-41d0-81d5-d05858b5bb35-kube-api-access-4qq7z\") pod \"e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") " pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:03.759154 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.759061    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-dshm\") pod \"e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") " pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:03.759154 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.759091    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0918ab9b-f101-41d0-81d5-d05858b5bb35-tls-certs\") pod \"e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") " pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:03.759154 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.759133    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-model-cache\") pod \"e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") " pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:03.759260 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.759173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-tmp-dir\") pod \"e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") " pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:03.759260 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.759197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-home\") pod \"e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") " pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:03.860528 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.860496    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-dshm\") pod \"e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") " pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:03.860528 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.860529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0918ab9b-f101-41d0-81d5-d05858b5bb35-tls-certs\") pod \"e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") " pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:03.860749 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.860551    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-model-cache\") pod \"e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") " pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:03.860749 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.860587    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-tmp-dir\") pod \"e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") " pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:03.860749 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.860605    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-home\") pod \"e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") " pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:03.860749 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.860628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qq7z\" (UniqueName: \"kubernetes.io/projected/0918ab9b-f101-41d0-81d5-d05858b5bb35-kube-api-access-4qq7z\") pod \"e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") " pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:03.861037 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.861016    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-tmp-dir\") pod \"e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") " pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:03.861118 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.861036    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-home\") pod \"e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") " pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:03.861118 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.861084    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-model-cache\") pod \"e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") " pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:03.862861 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.862835    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-dshm\") pod \"e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") " pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:03.862968 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.862934    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0918ab9b-f101-41d0-81d5-d05858b5bb35-tls-certs\") pod \"e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") " pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:03.873726 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.873697    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qq7z\" (UniqueName: \"kubernetes.io/projected/0918ab9b-f101-41d0-81d5-d05858b5bb35-kube-api-access-4qq7z\") pod \"e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") " pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:03.975882 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:03.975774    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:04.136111 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:04.136084    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"]
Jul 20 20:31:04.137600 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:31:04.137566    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0918ab9b_f101_41d0_81d5_d05858b5bb35.slice/crio-8788bb3df4e28b4f83af29e8f075cc37a7b77cc611a6ae6dd4a3963069082819 WatchSource:0}: Error finding container 8788bb3df4e28b4f83af29e8f075cc37a7b77cc611a6ae6dd4a3963069082819: Status 404 returned error can't find the container with id 8788bb3df4e28b4f83af29e8f075cc37a7b77cc611a6ae6dd4a3963069082819
Jul 20 20:31:04.711842 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:04.711803    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn" event={"ID":"0918ab9b-f101-41d0-81d5-d05858b5bb35","Type":"ContainerStarted","Data":"ab9b832b4dcc4de68e7ab5bba2c8cb3c8b672a990a34f99210e2b843284896da"}
Jul 20 20:31:04.712284 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:04.711850    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn" event={"ID":"0918ab9b-f101-41d0-81d5-d05858b5bb35","Type":"ContainerStarted","Data":"8788bb3df4e28b4f83af29e8f075cc37a7b77cc611a6ae6dd4a3963069082819"}
Jul 20 20:31:04.735468 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:04.735418    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn" podStartSLOduration=1.73540301 podStartE2EDuration="1.73540301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:31:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:31:04.734835699 +0000 UTC m=+2079.998905744" watchObservedRunningTime="2026-07-20 20:31:04.73540301 +0000 UTC m=+2079.999473079"
Jul 20 20:31:09.254733 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:09.254699    2580 patch_prober.go:28] interesting pod/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": read tcp 10.133.0.2:47320->10.133.0.42:15021: read: connection reset by peer" start-of-body=
Jul 20 20:31:09.255176 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:09.254783    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9" podUID="6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": read tcp 10.133.0.2:47320->10.133.0.42:15021: read: connection reset by peer"
Jul 20 20:31:09.737875 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:09.737839    2580 generic.go:356] "Generic (PLEG): container finished" podID="6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" containerID="392860fb648f0faa590dce13fd620f4d47b6b95ce72b0d65966cb2f24de75917" exitCode=0
Jul 20 20:31:09.738088 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:09.737903    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9" event={"ID":"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0","Type":"ContainerDied","Data":"392860fb648f0faa590dce13fd620f4d47b6b95ce72b0d65966cb2f24de75917"}
Jul 20 20:31:09.738306 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:09.738292    2580 scope.go:117] "RemoveContainer" containerID="392860fb648f0faa590dce13fd620f4d47b6b95ce72b0d65966cb2f24de75917"
Jul 20 20:31:09.739284 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:09.739255    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:31:09.739345 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:09.739325    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:31:09.739383 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:09.739356    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:31:10.744690 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:10.744656    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9" event={"ID":"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0","Type":"ContainerStarted","Data":"490679b2b5cd0124620843970a6d7bca5dfc15222b7551da15b9facd5865581b"}
Jul 20 20:31:11.320719 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:11.320682    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:31:11.320918 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:11.320887    2580 patch_prober.go:28] interesting pod/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 20 20:31:11.320983 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:11.320944    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9" podUID="6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 20 20:31:12.321368 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:12.321333    2580 patch_prober.go:28] interesting pod/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 20 20:31:12.321771 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:12.321389    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9" podUID="6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 20 20:31:13.320436 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:13.320387    2580 patch_prober.go:28] interesting pod/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 20 20:31:13.320617 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:13.320454    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9" podUID="6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 20 20:31:13.976086 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:13.976035    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:13.976974 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:13.976884    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:13.999415 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:13.999385    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:14.321255 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:14.321225    2580 patch_prober.go:28] interesting pod/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 20 20:31:14.321420 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:14.321298    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9" podUID="6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 20 20:31:14.775563 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:14.775478    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:31:15.324256 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:15.324226    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:31:15.324721 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:15.324559    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:31:15.325794 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:15.325778    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:31:25.188459 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:25.188429    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 20:31:25.195457 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:31:25.195433    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 20:32:10.576570 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:10.576536    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"]
Jul 20 20:32:25.273681 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:25.273652    2580 scope.go:117] "RemoveContainer" containerID="df8680503632afba12e8c29c304a8ea622090735852cf2520f9667d861291847"
Jul 20 20:32:25.281623 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:25.281597    2580 scope.go:117] "RemoveContainer" containerID="e1b0d5bbce45811b26708e6ef00635f62e71e4142d8c2fe61a8da4c6ee7ef620"
Jul 20 20:32:25.596264 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:25.596221    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn" podUID="0918ab9b-f101-41d0-81d5-d05858b5bb35" containerName="main" containerID="cri-o://ab9b832b4dcc4de68e7ab5bba2c8cb3c8b672a990a34f99210e2b843284896da" gracePeriod=45
Jul 20 20:32:25.843773 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:25.843750    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:32:25.868242 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:25.868168    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-home\") pod \"0918ab9b-f101-41d0-81d5-d05858b5bb35\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") "
Jul 20 20:32:25.868242 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:25.868222    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-tmp-dir\") pod \"0918ab9b-f101-41d0-81d5-d05858b5bb35\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") "
Jul 20 20:32:25.868436 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:25.868291    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qq7z\" (UniqueName: \"kubernetes.io/projected/0918ab9b-f101-41d0-81d5-d05858b5bb35-kube-api-access-4qq7z\") pod \"0918ab9b-f101-41d0-81d5-d05858b5bb35\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") "
Jul 20 20:32:25.868436 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:25.868316    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0918ab9b-f101-41d0-81d5-d05858b5bb35-tls-certs\") pod \"0918ab9b-f101-41d0-81d5-d05858b5bb35\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") "
Jul 20 20:32:25.868436 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:25.868355    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-dshm\") pod \"0918ab9b-f101-41d0-81d5-d05858b5bb35\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") "
Jul 20 20:32:25.868580 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:25.868464    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0918ab9b-f101-41d0-81d5-d05858b5bb35" (UID: "0918ab9b-f101-41d0-81d5-d05858b5bb35"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:32:25.868580 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:25.868498    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-model-cache\") pod \"0918ab9b-f101-41d0-81d5-d05858b5bb35\" (UID: \"0918ab9b-f101-41d0-81d5-d05858b5bb35\") "
Jul 20 20:32:25.868580 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:25.868545    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-home" (OuterVolumeSpecName: "home") pod "0918ab9b-f101-41d0-81d5-d05858b5bb35" (UID: "0918ab9b-f101-41d0-81d5-d05858b5bb35"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:32:25.868761 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:25.868743    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-home\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:32:25.868830 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:25.868756    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-model-cache" (OuterVolumeSpecName: "model-cache") pod "0918ab9b-f101-41d0-81d5-d05858b5bb35" (UID: "0918ab9b-f101-41d0-81d5-d05858b5bb35"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:32:25.868830 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:25.868769    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-tmp-dir\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:32:25.870757 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:25.870730    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-dshm" (OuterVolumeSpecName: "dshm") pod "0918ab9b-f101-41d0-81d5-d05858b5bb35" (UID: "0918ab9b-f101-41d0-81d5-d05858b5bb35"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:32:25.870757 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:25.870739    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0918ab9b-f101-41d0-81d5-d05858b5bb35-kube-api-access-4qq7z" (OuterVolumeSpecName: "kube-api-access-4qq7z") pod "0918ab9b-f101-41d0-81d5-d05858b5bb35" (UID: "0918ab9b-f101-41d0-81d5-d05858b5bb35"). InnerVolumeSpecName "kube-api-access-4qq7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:32:25.870938 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:25.870802    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0918ab9b-f101-41d0-81d5-d05858b5bb35-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0918ab9b-f101-41d0-81d5-d05858b5bb35" (UID: "0918ab9b-f101-41d0-81d5-d05858b5bb35"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 20:32:25.969284 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:25.969251    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qq7z\" (UniqueName: \"kubernetes.io/projected/0918ab9b-f101-41d0-81d5-d05858b5bb35-kube-api-access-4qq7z\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:32:25.969284 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:25.969279    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0918ab9b-f101-41d0-81d5-d05858b5bb35-tls-certs\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:32:25.969284 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:25.969288    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-dshm\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:32:25.969518 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:25.969297    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0918ab9b-f101-41d0-81d5-d05858b5bb35-model-cache\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:32:26.079466 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:26.079433    2580 generic.go:356] "Generic (PLEG): container finished" podID="0918ab9b-f101-41d0-81d5-d05858b5bb35" containerID="ab9b832b4dcc4de68e7ab5bba2c8cb3c8b672a990a34f99210e2b843284896da" exitCode=0
Jul 20 20:32:26.079650 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:26.079486    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"
Jul 20 20:32:26.079650 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:26.079520    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn" event={"ID":"0918ab9b-f101-41d0-81d5-d05858b5bb35","Type":"ContainerDied","Data":"ab9b832b4dcc4de68e7ab5bba2c8cb3c8b672a990a34f99210e2b843284896da"}
Jul 20 20:32:26.079650 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:26.079565    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn" event={"ID":"0918ab9b-f101-41d0-81d5-d05858b5bb35","Type":"ContainerDied","Data":"8788bb3df4e28b4f83af29e8f075cc37a7b77cc611a6ae6dd4a3963069082819"}
Jul 20 20:32:26.079650 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:26.079588    2580 scope.go:117] "RemoveContainer" containerID="ab9b832b4dcc4de68e7ab5bba2c8cb3c8b672a990a34f99210e2b843284896da"
Jul 20 20:32:26.093889 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:26.093868    2580 scope.go:117] "RemoveContainer" containerID="ab9b832b4dcc4de68e7ab5bba2c8cb3c8b672a990a34f99210e2b843284896da"
Jul 20 20:32:26.094175 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:32:26.094155    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab9b832b4dcc4de68e7ab5bba2c8cb3c8b672a990a34f99210e2b843284896da\": container with ID starting with ab9b832b4dcc4de68e7ab5bba2c8cb3c8b672a990a34f99210e2b843284896da not found: ID does not exist" containerID="ab9b832b4dcc4de68e7ab5bba2c8cb3c8b672a990a34f99210e2b843284896da"
Jul 20 20:32:26.094233 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:26.094185    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab9b832b4dcc4de68e7ab5bba2c8cb3c8b672a990a34f99210e2b843284896da"} err="failed to get container status \"ab9b832b4dcc4de68e7ab5bba2c8cb3c8b672a990a34f99210e2b843284896da\": rpc error: code = NotFound desc = could not find container \"ab9b832b4dcc4de68e7ab5bba2c8cb3c8b672a990a34f99210e2b843284896da\": container with ID starting with ab9b832b4dcc4de68e7ab5bba2c8cb3c8b672a990a34f99210e2b843284896da not found: ID does not exist"
Jul 20 20:32:26.104933 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:26.104905    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"]
Jul 20 20:32:26.108871 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:26.108848    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-fa45191d/e2e-lifecycle-model-fa45191d-kserve-f75647f9f-tfhcn"]
Jul 20 20:32:27.176437 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:32:27.176406    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0918ab9b-f101-41d0-81d5-d05858b5bb35" path="/var/lib/kubelet/pods/0918ab9b-f101-41d0-81d5-d05858b5bb35/volumes"
Jul 20 20:33:03.012760 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:03.012725    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"]
Jul 20 20:33:03.013469 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:03.013044    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9" podUID="6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" containerName="istio-proxy" containerID="cri-o://490679b2b5cd0124620843970a6d7bca5dfc15222b7551da15b9facd5865581b" gracePeriod=30
Jul 20 20:33:08.263534 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.263490    2580 generic.go:356] "Generic (PLEG): container finished" podID="6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" containerID="490679b2b5cd0124620843970a6d7bca5dfc15222b7551da15b9facd5865581b" exitCode=0
Jul 20 20:33:08.263891 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.263570    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9" event={"ID":"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0","Type":"ContainerDied","Data":"490679b2b5cd0124620843970a6d7bca5dfc15222b7551da15b9facd5865581b"}
Jul 20 20:33:08.263891 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.263643    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9" event={"ID":"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0","Type":"ContainerDied","Data":"25c12324582d889593399a10ecf4c100e0af881c86d8827eb7e6f2b77c1745a6"}
Jul 20 20:33:08.263891 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.263667    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25c12324582d889593399a10ecf4c100e0af881c86d8827eb7e6f2b77c1745a6"
Jul 20 20:33:08.263891 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.263687    2580 scope.go:117] "RemoveContainer" containerID="392860fb648f0faa590dce13fd620f4d47b6b95ce72b0d65966cb2f24de75917"
Jul 20 20:33:08.278118 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.278091    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:33:08.423608 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.423578    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-workload-certs\") pod \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") "
Jul 20 20:33:08.423608 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.423615    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istiod-ca-cert\") pod \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") "
Jul 20 20:33:08.423871 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.423650    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-data\") pod \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") "
Jul 20 20:33:08.423871 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.423677    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-envoy\") pod \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") "
Jul 20 20:33:08.423871 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.423709    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nklzp\" (UniqueName: \"kubernetes.io/projected/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-kube-api-access-nklzp\") pod \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") "
Jul 20 20:33:08.423871 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.423739    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-workload-socket\") pod \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") "
Jul 20 20:33:08.423871 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.423768    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-podinfo\") pod \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") "
Jul 20 20:33:08.423871 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.423792    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-credential-socket\") pod \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") "
Jul 20 20:33:08.423871 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.423820    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-token\") pod \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\" (UID: \"6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0\") "
Jul 20 20:33:08.423871 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.423832    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" (UID: "6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:33:08.424318 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.424153    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-workload-certs\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:33:08.424318 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.424235    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" (UID: "6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:33:08.424318 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.424260    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" (UID: "6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:33:08.424484 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.424426    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" (UID: "6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 20:33:08.424634 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.424597    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-data" (OuterVolumeSpecName: "istio-data") pod "6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" (UID: "6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:33:08.426293 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.426269    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" (UID: "6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 20:33:08.426413 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.426343    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-token" (OuterVolumeSpecName: "istio-token") pod "6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" (UID: "6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:33:08.426413 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.426363    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" (UID: "6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:33:08.426654 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.426636    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-kube-api-access-nklzp" (OuterVolumeSpecName: "kube-api-access-nklzp") pod "6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" (UID: "6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0"). InnerVolumeSpecName "kube-api-access-nklzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:33:08.524950 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.524861    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istiod-ca-cert\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:33:08.524950 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.524890    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-data\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:33:08.524950 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.524899    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-envoy\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:33:08.524950 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.524907    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nklzp\" (UniqueName: \"kubernetes.io/projected/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-kube-api-access-nklzp\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:33:08.524950 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.524917    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-workload-socket\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:33:08.524950 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.524926    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-podinfo\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:33:08.524950 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.524940    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-credential-socket\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:33:08.524950 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:08.524949    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0-istio-token\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:33:09.268680 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:09.268652    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"
Jul 20 20:33:09.304130 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:09.304100    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"]
Jul 20 20:33:09.307367 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:09.307335    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-fa45191d-openshift-default-67d7c8bf4c-wh6m9"]
Jul 20 20:33:11.175658 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:11.175615    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" path="/var/lib/kubelet/pods/6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0/volumes"
Jul 20 20:33:25.301391 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:25.301357    2580 scope.go:117] "RemoveContainer" containerID="202fdcad790ffbf6ad6f857f1a3393a96662aa0735f41bd74faa9a1530b47e2b"
Jul 20 20:33:59.784683 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.784602    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"]
Jul 20 20:33:59.785175 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.784890    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" containerName="istio-proxy"
Jul 20 20:33:59.785175 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.784901    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" containerName="istio-proxy"
Jul 20 20:33:59.785175 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.784917    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0918ab9b-f101-41d0-81d5-d05858b5bb35" containerName="main"
Jul 20 20:33:59.785175 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.784922    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0918ab9b-f101-41d0-81d5-d05858b5bb35" containerName="main"
Jul 20 20:33:59.785175 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.784930    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" containerName="istio-proxy"
Jul 20 20:33:59.785175 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.784946    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" containerName="istio-proxy"
Jul 20 20:33:59.785175 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.785013    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" containerName="istio-proxy"
Jul 20 20:33:59.785175 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.785021    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bc8e78f-68c4-4bbc-b9a7-34c956ac7ab0" containerName="istio-proxy"
Jul 20 20:33:59.785175 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.785028    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0918ab9b-f101-41d0-81d5-d05858b5bb35" containerName="main"
Jul 20 20:33:59.787900 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.787884    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.791215 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.791177    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 20 20:33:59.791215 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.791208    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-68af9a-openshift-default-dockercfg-5gdmv\""
Jul 20 20:33:59.808179 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.808157    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"]
Jul 20 20:33:59.867651 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.867609    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-token\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.867850 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.867662    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-credential-socket\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.867850 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.867742    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-podinfo\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.867850 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.867763    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-workload-certs\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.867850 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.867784    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gwzk\" (UniqueName: \"kubernetes.io/projected/56cf20d1-f7f1-4baf-9142-856dad37dc95-kube-api-access-8gwzk\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.867850 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.867828    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56cf20d1-f7f1-4baf-9142-856dad37dc95-istiod-ca-cert\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.868143 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.867870    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-workload-socket\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.868143 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.867888    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-envoy\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.868143 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.867922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-data\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.968357 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.968323    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-data\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.968551 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.968385    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-token\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.968551 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.968417    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-credential-socket\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.968551 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.968440    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-podinfo\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.968551 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.968462    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-workload-certs\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.968551 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.968488    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gwzk\" (UniqueName: \"kubernetes.io/projected/56cf20d1-f7f1-4baf-9142-856dad37dc95-kube-api-access-8gwzk\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.968829 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.968741    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56cf20d1-f7f1-4baf-9142-856dad37dc95-istiod-ca-cert\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.968829 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.968786    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-data\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.968829 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.968787    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-workload-socket\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.969026 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.968836    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-workload-certs\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.969026 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.968845    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-envoy\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.969026 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.968883    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-credential-socket\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.969145 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.969082    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-workload-socket\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.969512 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.969489    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56cf20d1-f7f1-4baf-9142-856dad37dc95-istiod-ca-cert\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.971048 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.971026    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-envoy\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.971111 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.971054    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-podinfo\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.978025 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.978000    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-token\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:33:59.978148 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:33:59.978086    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gwzk\" (UniqueName: \"kubernetes.io/projected/56cf20d1-f7f1-4baf-9142-856dad37dc95-kube-api-access-8gwzk\") pod \"e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") " pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:34:00.097300 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:00.097261    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:34:00.251107 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:00.251082    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"]
Jul 20 20:34:00.253019 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:34:00.252991    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56cf20d1_f7f1_4baf_9142_856dad37dc95.slice/crio-d8b50c8109db6707dd3b4dd273517d5f0487df5bab6d85a2001c837ce42fea32 WatchSource:0}: Error finding container d8b50c8109db6707dd3b4dd273517d5f0487df5bab6d85a2001c837ce42fea32: Status 404 returned error can't find the container with id d8b50c8109db6707dd3b4dd273517d5f0487df5bab6d85a2001c837ce42fea32
Jul 20 20:34:00.255133 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:00.255078    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:34:00.255255 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:00.255181    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:34:00.255255 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:00.255222    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:34:00.497623 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:00.497537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs" event={"ID":"56cf20d1-f7f1-4baf-9142-856dad37dc95","Type":"ContainerStarted","Data":"dd6127ad3059ab15debad619731ab0a4ac31b7096a2fc4f7ea0e8d7e7d7f809d"}
Jul 20 20:34:00.497623 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:00.497602    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs" event={"ID":"56cf20d1-f7f1-4baf-9142-856dad37dc95","Type":"ContainerStarted","Data":"d8b50c8109db6707dd3b4dd273517d5f0487df5bab6d85a2001c837ce42fea32"}
Jul 20 20:34:00.518818 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:00.518769    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs" podStartSLOduration=1.518752908 podStartE2EDuration="1.518752908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:33:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:34:00.5172085 +0000 UTC m=+2255.781278548" watchObservedRunningTime="2026-07-20 20:34:00.518752908 +0000 UTC m=+2255.782822954"
Jul 20 20:34:01.098069 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:01.098032    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:34:01.102762 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:01.102735    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:34:01.501818 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:01.501736    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:34:01.502723 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:01.502703    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:34:07.721837 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.721792    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"]
Jul 20 20:34:07.725090 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.725073    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.730049 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.730023    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-183471-openshift-default-dockercfg-zpvvl\""
Jul 20 20:34:07.739777 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.739751    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"]
Jul 20 20:34:07.838615 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.838581    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpsx7\" (UniqueName: \"kubernetes.io/projected/6737b3e1-0fc8-4de0-9cd6-451b193d957c-kube-api-access-lpsx7\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.838820 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.838631    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-workload-socket\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.838820 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.838655    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-podinfo\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.838820 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.838721    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-workload-certs\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.838820 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.838764    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-envoy\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.838820 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.838788    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-data\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.839093 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.838826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istiod-ca-cert\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.839093 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.838880    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-token\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.839093 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.838911    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-credential-socket\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.940098 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.940059    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-workload-socket\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.940098 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.940101    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-podinfo\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.940344 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.940131    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-workload-certs\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.940344 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.940149    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-envoy\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.940344 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.940265    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-data\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.940344 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.940328    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istiod-ca-cert\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.940546 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.940380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-token\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.940631 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.940607    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-workload-certs\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.940631 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.940616    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-credential-socket\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.940740 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.940658    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-workload-socket\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.940740 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.940699    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpsx7\" (UniqueName: \"kubernetes.io/projected/6737b3e1-0fc8-4de0-9cd6-451b193d957c-kube-api-access-lpsx7\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.941138 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.941117    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-data\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.941242 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.941168    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-credential-socket\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.941418 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.941396    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istiod-ca-cert\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.942466 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.942448    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-envoy\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.942675 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.942658    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-podinfo\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.956781 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.956753    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-token\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:07.956953 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:07.956924    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpsx7\" (UniqueName: \"kubernetes.io/projected/6737b3e1-0fc8-4de0-9cd6-451b193d957c-kube-api-access-lpsx7\") pod \"e2e-inf-b-183471-openshift-default-68f765f47-xw6sm\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") " pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:08.036052 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:08.035941    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:08.219258 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:08.219237    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"]
Jul 20 20:34:08.221361 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:34:08.221332    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6737b3e1_0fc8_4de0_9cd6_451b193d957c.slice/crio-84c758287603510836618e4604d915ab8984b94120bf5d8a237b0f47bccdb89d WatchSource:0}: Error finding container 84c758287603510836618e4604d915ab8984b94120bf5d8a237b0f47bccdb89d: Status 404 returned error can't find the container with id 84c758287603510836618e4604d915ab8984b94120bf5d8a237b0f47bccdb89d
Jul 20 20:34:08.223368 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:08.223337    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:34:08.223429 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:08.223401    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:34:08.223471 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:08.223430    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:34:08.538626 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:08.538593    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm" event={"ID":"6737b3e1-0fc8-4de0-9cd6-451b193d957c","Type":"ContainerStarted","Data":"c4d1d82ed74650067f3abd06c9e0e06dd6578b9f782e3ae5a88a09f1dacae7d3"}
Jul 20 20:34:08.538783 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:08.538652    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm" event={"ID":"6737b3e1-0fc8-4de0-9cd6-451b193d957c","Type":"ContainerStarted","Data":"84c758287603510836618e4604d915ab8984b94120bf5d8a237b0f47bccdb89d"}
Jul 20 20:34:08.562462 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:08.562416    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm" podStartSLOduration=1.562402141 podStartE2EDuration="1.562402141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:34:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:34:08.560790103 +0000 UTC m=+2263.824860150" watchObservedRunningTime="2026-07-20 20:34:08.562402141 +0000 UTC m=+2263.826472187"
Jul 20 20:34:09.036902 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:09.036816    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:09.041601 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:09.041576    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:09.543100 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:09.543073    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:09.544076 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:09.544054    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:14.566259 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:14.566227    2580 generic.go:356] "Generic (PLEG): container finished" podID="56cf20d1-f7f1-4baf-9142-856dad37dc95" containerID="dd6127ad3059ab15debad619731ab0a4ac31b7096a2fc4f7ea0e8d7e7d7f809d" exitCode=0
Jul 20 20:34:14.566678 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:14.566299    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs" event={"ID":"56cf20d1-f7f1-4baf-9142-856dad37dc95","Type":"ContainerDied","Data":"dd6127ad3059ab15debad619731ab0a4ac31b7096a2fc4f7ea0e8d7e7d7f809d"}
Jul 20 20:34:14.566678 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:14.566664    2580 scope.go:117] "RemoveContainer" containerID="dd6127ad3059ab15debad619731ab0a4ac31b7096a2fc4f7ea0e8d7e7d7f809d"
Jul 20 20:34:14.567649 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:14.567620    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:34:14.567753 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:14.567690    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:34:14.567753 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:14.567726    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:34:15.578342 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:15.577839    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs" event={"ID":"56cf20d1-f7f1-4baf-9142-856dad37dc95","Type":"ContainerStarted","Data":"95caa0d975a5a47739c4ec58d40cdf3b606d01ab811c7cc638fc4c632f197950"}
Jul 20 20:34:16.098266 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:16.098222    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:34:16.099591 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:16.099558    2580 patch_prober.go:28] interesting pod/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 20 20:34:16.099870 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:16.099826    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs" podUID="56cf20d1-f7f1-4baf-9142-856dad37dc95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 20 20:34:16.502036 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:16.501884    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:34:17.098632 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:17.098586    2580 patch_prober.go:28] interesting pod/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 20 20:34:17.099183 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:17.098655    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs" podUID="56cf20d1-f7f1-4baf-9142-856dad37dc95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 20 20:34:18.098985 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:18.098089    2580 patch_prober.go:28] interesting pod/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 20 20:34:18.098985 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:18.098170    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs" podUID="56cf20d1-f7f1-4baf-9142-856dad37dc95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 20 20:34:19.098518 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:19.098472    2580 patch_prober.go:28] interesting pod/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 20 20:34:19.098731 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:19.098552    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs" podUID="56cf20d1-f7f1-4baf-9142-856dad37dc95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 20 20:34:20.101859 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:20.101831    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:34:20.102934 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:20.102914    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:34:22.619506 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:22.619424    2580 generic.go:356] "Generic (PLEG): container finished" podID="6737b3e1-0fc8-4de0-9cd6-451b193d957c" containerID="c4d1d82ed74650067f3abd06c9e0e06dd6578b9f782e3ae5a88a09f1dacae7d3" exitCode=0
Jul 20 20:34:22.619506 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:22.619485    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm" event={"ID":"6737b3e1-0fc8-4de0-9cd6-451b193d957c","Type":"ContainerDied","Data":"c4d1d82ed74650067f3abd06c9e0e06dd6578b9f782e3ae5a88a09f1dacae7d3"}
Jul 20 20:34:22.619921 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:22.619850    2580 scope.go:117] "RemoveContainer" containerID="c4d1d82ed74650067f3abd06c9e0e06dd6578b9f782e3ae5a88a09f1dacae7d3"
Jul 20 20:34:22.620856 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:22.620827    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:34:22.620922 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:22.620893    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:34:22.620990 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:22.620927    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:34:23.625510 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:23.625475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm" event={"ID":"6737b3e1-0fc8-4de0-9cd6-451b193d957c","Type":"ContainerStarted","Data":"32d0d62e6df2b3060d060802e5ba714a6fdc523c2e6dfb49604fd8a14cd41835"}
Jul 20 20:34:24.037131 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:24.037045    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:24.038254 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:24.038226    2580 patch_prober.go:28] interesting pod/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 20 20:34:24.038384 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:24.038275    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm" podUID="6737b3e1-0fc8-4de0-9cd6-451b193d957c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 20 20:34:24.543840 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:24.543796    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:25.036642 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:25.036606    2580 patch_prober.go:28] interesting pod/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 20 20:34:25.037079 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:25.036662    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm" podUID="6737b3e1-0fc8-4de0-9cd6-451b193d957c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 20 20:34:25.322688 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:25.322660    2580 scope.go:117] "RemoveContainer" containerID="29674872e1df38156b2e6d3667085ea2b53fd46936cab4f9550c71d70511c5fc"
Jul 20 20:34:26.037326 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:26.037290    2580 patch_prober.go:28] interesting pod/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 20 20:34:26.037729 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:26.037348    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm" podUID="6737b3e1-0fc8-4de0-9cd6-451b193d957c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 20 20:34:27.037282 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:27.037239    2580 patch_prober.go:28] interesting pod/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 20 20:34:27.037469 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:27.037295    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm" podUID="6737b3e1-0fc8-4de0-9cd6-451b193d957c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 20 20:34:28.040319 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:28.040288    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:28.041169 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:28.041152    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:34:37.425861 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.425826    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"]
Jul 20 20:34:37.429083 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.429065    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:37.432180 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.432155    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-68af9a\"/\"kube-root-ca.crt\""
Jul 20 20:34:37.432293 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.432155    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-68af9a\"/\"default-dockercfg-dvbdp\""
Jul 20 20:34:37.433319 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.433299    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-68af9a\"/\"openshift-service-ca.crt\""
Jul 20 20:34:37.433430 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.433344    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-68af9a\"/\"test-model-68af9a-kserve-self-signed-certs\""
Jul 20 20:34:37.438970 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.438936    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"]
Jul 20 20:34:37.510389 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.510353    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-home\") pod \"test-model-68af9a-kserve-6d8d5dcf99-nzgcr\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") " pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:37.510389 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.510386    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-model-cache\") pod \"test-model-68af9a-kserve-6d8d5dcf99-nzgcr\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") " pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:37.510596 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.510419    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lxk7\" (UniqueName: \"kubernetes.io/projected/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-kube-api-access-7lxk7\") pod \"test-model-68af9a-kserve-6d8d5dcf99-nzgcr\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") " pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:37.510596 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.510469    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-dshm\") pod \"test-model-68af9a-kserve-6d8d5dcf99-nzgcr\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") " pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:37.510596 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.510493    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-tmp-dir\") pod \"test-model-68af9a-kserve-6d8d5dcf99-nzgcr\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") " pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:37.510596 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.510566    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-tls-certs\") pod \"test-model-68af9a-kserve-6d8d5dcf99-nzgcr\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") " pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:37.611113 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.611077    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-dshm\") pod \"test-model-68af9a-kserve-6d8d5dcf99-nzgcr\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") " pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:37.611113 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.611112    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-tmp-dir\") pod \"test-model-68af9a-kserve-6d8d5dcf99-nzgcr\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") " pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:37.611327 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.611148    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-tls-certs\") pod \"test-model-68af9a-kserve-6d8d5dcf99-nzgcr\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") " pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:37.611327 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.611306    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-home\") pod \"test-model-68af9a-kserve-6d8d5dcf99-nzgcr\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") " pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:37.611428 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.611339    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-model-cache\") pod \"test-model-68af9a-kserve-6d8d5dcf99-nzgcr\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") " pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:37.611428 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.611380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lxk7\" (UniqueName: \"kubernetes.io/projected/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-kube-api-access-7lxk7\") pod \"test-model-68af9a-kserve-6d8d5dcf99-nzgcr\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") " pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:37.611590 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.611565    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-tmp-dir\") pod \"test-model-68af9a-kserve-6d8d5dcf99-nzgcr\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") " pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:37.611665 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.611645    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-home\") pod \"test-model-68af9a-kserve-6d8d5dcf99-nzgcr\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") " pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:37.611711 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.611692    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-model-cache\") pod \"test-model-68af9a-kserve-6d8d5dcf99-nzgcr\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") " pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:37.613425 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.613395    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-dshm\") pod \"test-model-68af9a-kserve-6d8d5dcf99-nzgcr\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") " pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:37.613596 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.613579    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-tls-certs\") pod \"test-model-68af9a-kserve-6d8d5dcf99-nzgcr\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") " pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:37.620831 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.620807    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lxk7\" (UniqueName: \"kubernetes.io/projected/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-kube-api-access-7lxk7\") pod \"test-model-68af9a-kserve-6d8d5dcf99-nzgcr\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") " pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:37.739353 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.739261    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:37.869750 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:37.869726    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"]
Jul 20 20:34:37.872339 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:34:37.872311    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5ba5dce_dcb8_4c3c_a79c_fb43d2c3f628.slice/crio-8e9516dd7674a42cc57c0ff44176140940858b54927c5485d6ead076b4e41283 WatchSource:0}: Error finding container 8e9516dd7674a42cc57c0ff44176140940858b54927c5485d6ead076b4e41283: Status 404 returned error can't find the container with id 8e9516dd7674a42cc57c0ff44176140940858b54927c5485d6ead076b4e41283
Jul 20 20:34:38.692505 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:38.692466    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr" event={"ID":"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628","Type":"ContainerStarted","Data":"68b21be248b46c3e55623fb8936f4329edb88ef0b2107002144b7958d9118a86"}
Jul 20 20:34:38.693089 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:38.692514    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr" event={"ID":"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628","Type":"ContainerStarted","Data":"8e9516dd7674a42cc57c0ff44176140940858b54927c5485d6ead076b4e41283"}
Jul 20 20:34:47.739612 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:47.739575    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:47.740025 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:47.739628    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:47.752163 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:47.752140    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:34:47.770269 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:47.770228    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr" podStartSLOduration=10.770211884 podStartE2EDuration="10.770211884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:34:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:34:38.711370118 +0000 UTC m=+2293.975440166" watchObservedRunningTime="2026-07-20 20:34:47.770211884 +0000 UTC m=+2303.034281931"
Jul 20 20:34:48.745801 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:34:48.745774    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:35:05.635323 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.635296    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"]
Jul 20 20:35:05.638415 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.638395    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:05.641149 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.641125    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-183471\"/\"kube-root-ca.crt\""
Jul 20 20:35:05.641278 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.641178    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-183471\"/\"openshift-service-ca.crt\""
Jul 20 20:35:05.642547 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.642521    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-183471\"/\"test-model-183471-kserve-self-signed-certs\""
Jul 20 20:35:05.642669 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.642591    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-183471\"/\"default-dockercfg-7mcm4\""
Jul 20 20:35:05.648569 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.648543    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"]
Jul 20 20:35:05.650130 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.650111    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-model-cache\") pod \"test-model-183471-kserve-f79fd7b69-mqtl4\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") " pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:05.650254 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.650163    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-tmp-dir\") pod \"test-model-183471-kserve-f79fd7b69-mqtl4\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") " pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:05.650254 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.650243    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36461305-ac08-4124-b300-d7aeb0e41fe5-tls-certs\") pod \"test-model-183471-kserve-f79fd7b69-mqtl4\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") " pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:05.650371 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.650322    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-home\") pod \"test-model-183471-kserve-f79fd7b69-mqtl4\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") " pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:05.650371 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.650360    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-dshm\") pod \"test-model-183471-kserve-f79fd7b69-mqtl4\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") " pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:05.650479 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.650389    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g65p8\" (UniqueName: \"kubernetes.io/projected/36461305-ac08-4124-b300-d7aeb0e41fe5-kube-api-access-g65p8\") pod \"test-model-183471-kserve-f79fd7b69-mqtl4\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") " pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:05.751217 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.751185    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-dshm\") pod \"test-model-183471-kserve-f79fd7b69-mqtl4\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") " pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:05.751217 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.751223    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g65p8\" (UniqueName: \"kubernetes.io/projected/36461305-ac08-4124-b300-d7aeb0e41fe5-kube-api-access-g65p8\") pod \"test-model-183471-kserve-f79fd7b69-mqtl4\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") " pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:05.751448 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.751350    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-model-cache\") pod \"test-model-183471-kserve-f79fd7b69-mqtl4\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") " pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:05.751448 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.751387    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-tmp-dir\") pod \"test-model-183471-kserve-f79fd7b69-mqtl4\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") " pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:05.751531 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.751456    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36461305-ac08-4124-b300-d7aeb0e41fe5-tls-certs\") pod \"test-model-183471-kserve-f79fd7b69-mqtl4\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") " pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:05.751531 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.751505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-home\") pod \"test-model-183471-kserve-f79fd7b69-mqtl4\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") " pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:05.751770 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.751747    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-model-cache\") pod \"test-model-183471-kserve-f79fd7b69-mqtl4\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") " pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:05.751838 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.751768    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-tmp-dir\") pod \"test-model-183471-kserve-f79fd7b69-mqtl4\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") " pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:05.751838 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.751803    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-home\") pod \"test-model-183471-kserve-f79fd7b69-mqtl4\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") " pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:05.753625 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.753602    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-dshm\") pod \"test-model-183471-kserve-f79fd7b69-mqtl4\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") " pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:05.753792 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.753776    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36461305-ac08-4124-b300-d7aeb0e41fe5-tls-certs\") pod \"test-model-183471-kserve-f79fd7b69-mqtl4\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") " pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:05.759813 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.759789    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g65p8\" (UniqueName: \"kubernetes.io/projected/36461305-ac08-4124-b300-d7aeb0e41fe5-kube-api-access-g65p8\") pod \"test-model-183471-kserve-f79fd7b69-mqtl4\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") " pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:05.950658 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:05.950577    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:06.085712 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:06.085681    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"]
Jul 20 20:35:06.088864 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:35:06.088837    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36461305_ac08_4124_b300_d7aeb0e41fe5.slice/crio-878d12330907fb4eb17060e65cf9bdaaac75b4d514a70177ea6020459b41ccd3 WatchSource:0}: Error finding container 878d12330907fb4eb17060e65cf9bdaaac75b4d514a70177ea6020459b41ccd3: Status 404 returned error can't find the container with id 878d12330907fb4eb17060e65cf9bdaaac75b4d514a70177ea6020459b41ccd3
Jul 20 20:35:06.816037 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:06.815998    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4" event={"ID":"36461305-ac08-4124-b300-d7aeb0e41fe5","Type":"ContainerStarted","Data":"e2a1496e50f3b91f6f8a511b323780ff399bfe9af2435df9b9e3ede4fb477b46"}
Jul 20 20:35:06.816037 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:06.816042    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4" event={"ID":"36461305-ac08-4124-b300-d7aeb0e41fe5","Type":"ContainerStarted","Data":"878d12330907fb4eb17060e65cf9bdaaac75b4d514a70177ea6020459b41ccd3"}
Jul 20 20:35:06.836599 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:06.836551    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4" podStartSLOduration=1.836535118 podStartE2EDuration="1.836535118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:35:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:35:06.833855868 +0000 UTC m=+2322.097925916" watchObservedRunningTime="2026-07-20 20:35:06.836535118 +0000 UTC m=+2322.100605163"
Jul 20 20:35:15.951507 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:15.951474    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:15.952125 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:15.951522    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:15.964233 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:15.964208    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:16.873616 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:16.873585    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:35:45.121579 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:45.121541    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"]
Jul 20 20:35:46.021146 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:35:46.021117    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"]
Jul 20 20:36:00.141840 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:00.141774    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr" podUID="a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628" containerName="main" containerID="cri-o://68b21be248b46c3e55623fb8936f4329edb88ef0b2107002144b7958d9118a86" gracePeriod=45
Jul 20 20:36:00.390667 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:00.390645    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:36:00.434338 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:00.434267    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lxk7\" (UniqueName: \"kubernetes.io/projected/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-kube-api-access-7lxk7\") pod \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") "
Jul 20 20:36:00.434338 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:00.434312    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-model-cache\") pod \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") "
Jul 20 20:36:00.434527 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:00.434349    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-tls-certs\") pod \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") "
Jul 20 20:36:00.434527 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:00.434378    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-home\") pod \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") "
Jul 20 20:36:00.434527 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:00.434394    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-tmp-dir\") pod \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") "
Jul 20 20:36:00.434527 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:00.434413    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-dshm\") pod \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\" (UID: \"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628\") "
Jul 20 20:36:00.434726 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:00.434647    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-model-cache" (OuterVolumeSpecName: "model-cache") pod "a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628" (UID: "a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:36:00.434726 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:00.434657    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-home" (OuterVolumeSpecName: "home") pod "a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628" (UID: "a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:36:00.434726 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:00.434672    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628" (UID: "a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:36:00.436510 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:00.436486    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-dshm" (OuterVolumeSpecName: "dshm") pod "a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628" (UID: "a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:36:00.436583 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:00.436545    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628" (UID: "a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 20:36:00.436664 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:00.436644    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-kube-api-access-7lxk7" (OuterVolumeSpecName: "kube-api-access-7lxk7") pod "a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628" (UID: "a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628"). InnerVolumeSpecName "kube-api-access-7lxk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:36:00.535951 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:00.535922    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lxk7\" (UniqueName: \"kubernetes.io/projected/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-kube-api-access-7lxk7\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:00.535951 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:00.535946    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-model-cache\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:00.535951 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:00.535974    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-tls-certs\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:00.536180 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:00.535982    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-home\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:00.536180 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:00.535990    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-tmp-dir\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:00.536180 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:00.535997    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628-dshm\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:01.041156 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.041089    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4" podUID="36461305-ac08-4124-b300-d7aeb0e41fe5" containerName="main" containerID="cri-o://e2a1496e50f3b91f6f8a511b323780ff399bfe9af2435df9b9e3ede4fb477b46" gracePeriod=45
Jul 20 20:36:01.060745 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.060719    2580 generic.go:356] "Generic (PLEG): container finished" podID="a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628" containerID="68b21be248b46c3e55623fb8936f4329edb88ef0b2107002144b7958d9118a86" exitCode=0
Jul 20 20:36:01.060850 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.060770    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"
Jul 20 20:36:01.060850 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.060780    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr" event={"ID":"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628","Type":"ContainerDied","Data":"68b21be248b46c3e55623fb8936f4329edb88ef0b2107002144b7958d9118a86"}
Jul 20 20:36:01.060850 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.060811    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr" event={"ID":"a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628","Type":"ContainerDied","Data":"8e9516dd7674a42cc57c0ff44176140940858b54927c5485d6ead076b4e41283"}
Jul 20 20:36:01.060850 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.060827    2580 scope.go:117] "RemoveContainer" containerID="68b21be248b46c3e55623fb8936f4329edb88ef0b2107002144b7958d9118a86"
Jul 20 20:36:01.116829 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.116795    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"]
Jul 20 20:36:01.120665 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.120642    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-68af9a/test-model-68af9a-kserve-6d8d5dcf99-nzgcr"]
Jul 20 20:36:01.153414 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.153238    2580 scope.go:117] "RemoveContainer" containerID="68b21be248b46c3e55623fb8936f4329edb88ef0b2107002144b7958d9118a86"
Jul 20 20:36:01.153693 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:36:01.153618    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68b21be248b46c3e55623fb8936f4329edb88ef0b2107002144b7958d9118a86\": container with ID starting with 68b21be248b46c3e55623fb8936f4329edb88ef0b2107002144b7958d9118a86 not found: ID does not exist" containerID="68b21be248b46c3e55623fb8936f4329edb88ef0b2107002144b7958d9118a86"
Jul 20 20:36:01.153693 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.153657    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68b21be248b46c3e55623fb8936f4329edb88ef0b2107002144b7958d9118a86"} err="failed to get container status \"68b21be248b46c3e55623fb8936f4329edb88ef0b2107002144b7958d9118a86\": rpc error: code = NotFound desc = could not find container \"68b21be248b46c3e55623fb8936f4329edb88ef0b2107002144b7958d9118a86\": container with ID starting with 68b21be248b46c3e55623fb8936f4329edb88ef0b2107002144b7958d9118a86 not found: ID does not exist"
Jul 20 20:36:01.176332 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.176299    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628" path="/var/lib/kubelet/pods/a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628/volumes"
Jul 20 20:36:01.294381 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.294315    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:36:01.343512 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.343483    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-tmp-dir\") pod \"36461305-ac08-4124-b300-d7aeb0e41fe5\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") "
Jul 20 20:36:01.343512 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.343525    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-model-cache\") pod \"36461305-ac08-4124-b300-d7aeb0e41fe5\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") "
Jul 20 20:36:01.343764 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.343548    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g65p8\" (UniqueName: \"kubernetes.io/projected/36461305-ac08-4124-b300-d7aeb0e41fe5-kube-api-access-g65p8\") pod \"36461305-ac08-4124-b300-d7aeb0e41fe5\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") "
Jul 20 20:36:01.343764 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.343595    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36461305-ac08-4124-b300-d7aeb0e41fe5-tls-certs\") pod \"36461305-ac08-4124-b300-d7aeb0e41fe5\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") "
Jul 20 20:36:01.343764 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.343637    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-dshm\") pod \"36461305-ac08-4124-b300-d7aeb0e41fe5\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") "
Jul 20 20:36:01.343764 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.343655    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-home\") pod \"36461305-ac08-4124-b300-d7aeb0e41fe5\" (UID: \"36461305-ac08-4124-b300-d7aeb0e41fe5\") "
Jul 20 20:36:01.343990 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.343817    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-model-cache" (OuterVolumeSpecName: "model-cache") pod "36461305-ac08-4124-b300-d7aeb0e41fe5" (UID: "36461305-ac08-4124-b300-d7aeb0e41fe5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:36:01.343990 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.343827    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "36461305-ac08-4124-b300-d7aeb0e41fe5" (UID: "36461305-ac08-4124-b300-d7aeb0e41fe5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:36:01.343990 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.343981    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-home" (OuterVolumeSpecName: "home") pod "36461305-ac08-4124-b300-d7aeb0e41fe5" (UID: "36461305-ac08-4124-b300-d7aeb0e41fe5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:36:01.345869 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.345839    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-dshm" (OuterVolumeSpecName: "dshm") pod "36461305-ac08-4124-b300-d7aeb0e41fe5" (UID: "36461305-ac08-4124-b300-d7aeb0e41fe5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:36:01.345998 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.345916    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36461305-ac08-4124-b300-d7aeb0e41fe5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "36461305-ac08-4124-b300-d7aeb0e41fe5" (UID: "36461305-ac08-4124-b300-d7aeb0e41fe5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 20:36:01.345998 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.345929    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36461305-ac08-4124-b300-d7aeb0e41fe5-kube-api-access-g65p8" (OuterVolumeSpecName: "kube-api-access-g65p8") pod "36461305-ac08-4124-b300-d7aeb0e41fe5" (UID: "36461305-ac08-4124-b300-d7aeb0e41fe5"). InnerVolumeSpecName "kube-api-access-g65p8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:36:01.444663 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.444615    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-dshm\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:01.444663 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.444657    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-home\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:01.444663 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.444667    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-tmp-dir\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:01.444663 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.444678    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36461305-ac08-4124-b300-d7aeb0e41fe5-model-cache\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:01.444663 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.444686    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g65p8\" (UniqueName: \"kubernetes.io/projected/36461305-ac08-4124-b300-d7aeb0e41fe5-kube-api-access-g65p8\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:01.445018 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:01.444695    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36461305-ac08-4124-b300-d7aeb0e41fe5-tls-certs\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:02.067444 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:02.067414    2580 generic.go:356] "Generic (PLEG): container finished" podID="36461305-ac08-4124-b300-d7aeb0e41fe5" containerID="e2a1496e50f3b91f6f8a511b323780ff399bfe9af2435df9b9e3ede4fb477b46" exitCode=0
Jul 20 20:36:02.067636 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:02.067455    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"
Jul 20 20:36:02.067636 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:02.067475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4" event={"ID":"36461305-ac08-4124-b300-d7aeb0e41fe5","Type":"ContainerDied","Data":"e2a1496e50f3b91f6f8a511b323780ff399bfe9af2435df9b9e3ede4fb477b46"}
Jul 20 20:36:02.067636 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:02.067506    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4" event={"ID":"36461305-ac08-4124-b300-d7aeb0e41fe5","Type":"ContainerDied","Data":"878d12330907fb4eb17060e65cf9bdaaac75b4d514a70177ea6020459b41ccd3"}
Jul 20 20:36:02.067636 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:02.067522    2580 scope.go:117] "RemoveContainer" containerID="e2a1496e50f3b91f6f8a511b323780ff399bfe9af2435df9b9e3ede4fb477b46"
Jul 20 20:36:02.079025 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:02.077533    2580 scope.go:117] "RemoveContainer" containerID="e2a1496e50f3b91f6f8a511b323780ff399bfe9af2435df9b9e3ede4fb477b46"
Jul 20 20:36:02.079356 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:36:02.079332    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2a1496e50f3b91f6f8a511b323780ff399bfe9af2435df9b9e3ede4fb477b46\": container with ID starting with e2a1496e50f3b91f6f8a511b323780ff399bfe9af2435df9b9e3ede4fb477b46 not found: ID does not exist" containerID="e2a1496e50f3b91f6f8a511b323780ff399bfe9af2435df9b9e3ede4fb477b46"
Jul 20 20:36:02.079458 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:02.079362    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2a1496e50f3b91f6f8a511b323780ff399bfe9af2435df9b9e3ede4fb477b46"} err="failed to get container status \"e2a1496e50f3b91f6f8a511b323780ff399bfe9af2435df9b9e3ede4fb477b46\": rpc error: code = NotFound desc = could not find container \"e2a1496e50f3b91f6f8a511b323780ff399bfe9af2435df9b9e3ede4fb477b46\": container with ID starting with e2a1496e50f3b91f6f8a511b323780ff399bfe9af2435df9b9e3ede4fb477b46 not found: ID does not exist"
Jul 20 20:36:02.089713 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:02.089686    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"]
Jul 20 20:36:02.093430 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:02.093405    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-183471/test-model-183471-kserve-f79fd7b69-mqtl4"]
Jul 20 20:36:03.175977 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:03.175936    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36461305-ac08-4124-b300-d7aeb0e41fe5" path="/var/lib/kubelet/pods/36461305-ac08-4124-b300-d7aeb0e41fe5/volumes"
Jul 20 20:36:08.090731 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:08.090693    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"]
Jul 20 20:36:08.091307 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:08.090904    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs" podUID="56cf20d1-f7f1-4baf-9142-856dad37dc95" containerName="istio-proxy" containerID="cri-o://95caa0d975a5a47739c4ec58d40cdf3b606d01ab811c7cc638fc4c632f197950" gracePeriod=30
Jul 20 20:36:13.336831 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.336805    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:36:13.451227 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.451129    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-token\") pod \"56cf20d1-f7f1-4baf-9142-856dad37dc95\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") "
Jul 20 20:36:13.451227 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.451174    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56cf20d1-f7f1-4baf-9142-856dad37dc95-istiod-ca-cert\") pod \"56cf20d1-f7f1-4baf-9142-856dad37dc95\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") "
Jul 20 20:36:13.451227 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.451197    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-data\") pod \"56cf20d1-f7f1-4baf-9142-856dad37dc95\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") "
Jul 20 20:36:13.451227 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.451227    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-credential-socket\") pod \"56cf20d1-f7f1-4baf-9142-856dad37dc95\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") "
Jul 20 20:36:13.451566 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.451245    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gwzk\" (UniqueName: \"kubernetes.io/projected/56cf20d1-f7f1-4baf-9142-856dad37dc95-kube-api-access-8gwzk\") pod \"56cf20d1-f7f1-4baf-9142-856dad37dc95\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") "
Jul 20 20:36:13.451566 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.451322    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-workload-certs\") pod \"56cf20d1-f7f1-4baf-9142-856dad37dc95\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") "
Jul 20 20:36:13.451566 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.451344    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-workload-socket\") pod \"56cf20d1-f7f1-4baf-9142-856dad37dc95\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") "
Jul 20 20:36:13.451566 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.451367    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-envoy\") pod \"56cf20d1-f7f1-4baf-9142-856dad37dc95\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") "
Jul 20 20:36:13.451566 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.451404    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-podinfo\") pod \"56cf20d1-f7f1-4baf-9142-856dad37dc95\" (UID: \"56cf20d1-f7f1-4baf-9142-856dad37dc95\") "
Jul 20 20:36:13.451832 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.451577    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56cf20d1-f7f1-4baf-9142-856dad37dc95-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "56cf20d1-f7f1-4baf-9142-856dad37dc95" (UID: "56cf20d1-f7f1-4baf-9142-856dad37dc95"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 20:36:13.451832 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.451663    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56cf20d1-f7f1-4baf-9142-856dad37dc95-istiod-ca-cert\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:13.451832 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.451687    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "56cf20d1-f7f1-4baf-9142-856dad37dc95" (UID: "56cf20d1-f7f1-4baf-9142-856dad37dc95"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:36:13.451832 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.451801    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "56cf20d1-f7f1-4baf-9142-856dad37dc95" (UID: "56cf20d1-f7f1-4baf-9142-856dad37dc95"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:36:13.452070 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.451868    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "56cf20d1-f7f1-4baf-9142-856dad37dc95" (UID: "56cf20d1-f7f1-4baf-9142-856dad37dc95"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:36:13.452325 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.452301    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-data" (OuterVolumeSpecName: "istio-data") pod "56cf20d1-f7f1-4baf-9142-856dad37dc95" (UID: "56cf20d1-f7f1-4baf-9142-856dad37dc95"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:36:13.453427 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.453398    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-token" (OuterVolumeSpecName: "istio-token") pod "56cf20d1-f7f1-4baf-9142-856dad37dc95" (UID: "56cf20d1-f7f1-4baf-9142-856dad37dc95"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:36:13.453513 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.453467    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56cf20d1-f7f1-4baf-9142-856dad37dc95-kube-api-access-8gwzk" (OuterVolumeSpecName: "kube-api-access-8gwzk") pod "56cf20d1-f7f1-4baf-9142-856dad37dc95" (UID: "56cf20d1-f7f1-4baf-9142-856dad37dc95"). InnerVolumeSpecName "kube-api-access-8gwzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:36:13.453566 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.453552    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "56cf20d1-f7f1-4baf-9142-856dad37dc95" (UID: "56cf20d1-f7f1-4baf-9142-856dad37dc95"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:36:13.453668 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.453654    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "56cf20d1-f7f1-4baf-9142-856dad37dc95" (UID: "56cf20d1-f7f1-4baf-9142-856dad37dc95"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 20:36:13.552803 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.552755    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-workload-certs\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:13.552803 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.552795    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-workload-socket\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:13.552803 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.552808    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-envoy\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:13.553113 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.552819    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-podinfo\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:13.553113 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.552831    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-token\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:13.553113 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.552841    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-istio-data\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:13.553113 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.552852    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56cf20d1-f7f1-4baf-9142-856dad37dc95-credential-socket\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:13.553113 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:13.552863    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gwzk\" (UniqueName: \"kubernetes.io/projected/56cf20d1-f7f1-4baf-9142-856dad37dc95-kube-api-access-8gwzk\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:14.125216 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:14.125174    2580 generic.go:356] "Generic (PLEG): container finished" podID="56cf20d1-f7f1-4baf-9142-856dad37dc95" containerID="95caa0d975a5a47739c4ec58d40cdf3b606d01ab811c7cc638fc4c632f197950" exitCode=0
Jul 20 20:36:14.125393 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:14.125227    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"
Jul 20 20:36:14.125393 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:14.125235    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs" event={"ID":"56cf20d1-f7f1-4baf-9142-856dad37dc95","Type":"ContainerDied","Data":"95caa0d975a5a47739c4ec58d40cdf3b606d01ab811c7cc638fc4c632f197950"}
Jul 20 20:36:14.125393 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:14.125291    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs" event={"ID":"56cf20d1-f7f1-4baf-9142-856dad37dc95","Type":"ContainerDied","Data":"d8b50c8109db6707dd3b4dd273517d5f0487df5bab6d85a2001c837ce42fea32"}
Jul 20 20:36:14.125393 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:14.125321    2580 scope.go:117] "RemoveContainer" containerID="95caa0d975a5a47739c4ec58d40cdf3b606d01ab811c7cc638fc4c632f197950"
Jul 20 20:36:14.134814 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:14.134794    2580 scope.go:117] "RemoveContainer" containerID="dd6127ad3059ab15debad619731ab0a4ac31b7096a2fc4f7ea0e8d7e7d7f809d"
Jul 20 20:36:14.142541 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:14.142524    2580 scope.go:117] "RemoveContainer" containerID="95caa0d975a5a47739c4ec58d40cdf3b606d01ab811c7cc638fc4c632f197950"
Jul 20 20:36:14.142786 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:36:14.142770    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95caa0d975a5a47739c4ec58d40cdf3b606d01ab811c7cc638fc4c632f197950\": container with ID starting with 95caa0d975a5a47739c4ec58d40cdf3b606d01ab811c7cc638fc4c632f197950 not found: ID does not exist" containerID="95caa0d975a5a47739c4ec58d40cdf3b606d01ab811c7cc638fc4c632f197950"
Jul 20 20:36:14.142828 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:14.142794    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95caa0d975a5a47739c4ec58d40cdf3b606d01ab811c7cc638fc4c632f197950"} err="failed to get container status \"95caa0d975a5a47739c4ec58d40cdf3b606d01ab811c7cc638fc4c632f197950\": rpc error: code = NotFound desc = could not find container \"95caa0d975a5a47739c4ec58d40cdf3b606d01ab811c7cc638fc4c632f197950\": container with ID starting with 95caa0d975a5a47739c4ec58d40cdf3b606d01ab811c7cc638fc4c632f197950 not found: ID does not exist"
Jul 20 20:36:14.142828 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:14.142813    2580 scope.go:117] "RemoveContainer" containerID="dd6127ad3059ab15debad619731ab0a4ac31b7096a2fc4f7ea0e8d7e7d7f809d"
Jul 20 20:36:14.143022 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:36:14.143006    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd6127ad3059ab15debad619731ab0a4ac31b7096a2fc4f7ea0e8d7e7d7f809d\": container with ID starting with dd6127ad3059ab15debad619731ab0a4ac31b7096a2fc4f7ea0e8d7e7d7f809d not found: ID does not exist" containerID="dd6127ad3059ab15debad619731ab0a4ac31b7096a2fc4f7ea0e8d7e7d7f809d"
Jul 20 20:36:14.143068 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:14.143027    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd6127ad3059ab15debad619731ab0a4ac31b7096a2fc4f7ea0e8d7e7d7f809d"} err="failed to get container status \"dd6127ad3059ab15debad619731ab0a4ac31b7096a2fc4f7ea0e8d7e7d7f809d\": rpc error: code = NotFound desc = could not find container \"dd6127ad3059ab15debad619731ab0a4ac31b7096a2fc4f7ea0e8d7e7d7f809d\": container with ID starting with dd6127ad3059ab15debad619731ab0a4ac31b7096a2fc4f7ea0e8d7e7d7f809d not found: ID does not exist"
Jul 20 20:36:14.148045 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:14.148021    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"]
Jul 20 20:36:14.151508 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:14.151485    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-68af9a-openshift-default-77c6766f9f-b2fgs"]
Jul 20 20:36:15.175742 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:15.175711    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56cf20d1-f7f1-4baf-9142-856dad37dc95" path="/var/lib/kubelet/pods/56cf20d1-f7f1-4baf-9142-856dad37dc95/volumes"
Jul 20 20:36:25.211396 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:25.211368    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 20:36:25.219786 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:25.219765    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 20:36:25.346880 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:25.346844    2580 scope.go:117] "RemoveContainer" containerID="76c4fa1d9ee1f8d1249450cc31ec43de570da20c96f8c92aa5a89f984e25cf74"
Jul 20 20:36:25.354788 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:25.354766    2580 scope.go:117] "RemoveContainer" containerID="452e01e8115799fedf40b333e25aa2f6456dc1f6cce45b5ed6633ad25239e9b1"
Jul 20 20:36:30.512264 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:30.512228    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"]
Jul 20 20:36:30.512689 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:30.512419    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm" podUID="6737b3e1-0fc8-4de0-9cd6-451b193d957c" containerName="istio-proxy" containerID="cri-o://32d0d62e6df2b3060d060802e5ba714a6fdc523c2e6dfb49604fd8a14cd41835" gracePeriod=30
Jul 20 20:36:31.365218 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.365180    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"]
Jul 20 20:36:31.365555 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.365538    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56cf20d1-f7f1-4baf-9142-856dad37dc95" containerName="istio-proxy"
Jul 20 20:36:31.365622 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.365558    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="56cf20d1-f7f1-4baf-9142-856dad37dc95" containerName="istio-proxy"
Jul 20 20:36:31.365622 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.365573    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36461305-ac08-4124-b300-d7aeb0e41fe5" containerName="main"
Jul 20 20:36:31.365622 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.365581    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="36461305-ac08-4124-b300-d7aeb0e41fe5" containerName="main"
Jul 20 20:36:31.365622 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.365593    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56cf20d1-f7f1-4baf-9142-856dad37dc95" containerName="istio-proxy"
Jul 20 20:36:31.365622 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.365603    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="56cf20d1-f7f1-4baf-9142-856dad37dc95" containerName="istio-proxy"
Jul 20 20:36:31.365879 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.365632    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628" containerName="main"
Jul 20 20:36:31.365879 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.365641    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628" containerName="main"
Jul 20 20:36:31.365879 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.365721    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="56cf20d1-f7f1-4baf-9142-856dad37dc95" containerName="istio-proxy"
Jul 20 20:36:31.365879 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.365734    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="36461305-ac08-4124-b300-d7aeb0e41fe5" containerName="main"
Jul 20 20:36:31.365879 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.365790    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5ba5dce-dcb8-4c3c-a79c-fb43d2c3f628" containerName="main"
Jul 20 20:36:31.365879 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.365800    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="56cf20d1-f7f1-4baf-9142-856dad37dc95" containerName="istio-proxy"
Jul 20 20:36:31.370527 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.370507    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.373221 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.373197    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-93c18a-openshift-default-dockercfg-446gr\""
Jul 20 20:36:31.381098 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.381073    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"]
Jul 20 20:36:31.506468 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.506428    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-envoy\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.506644 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.506482    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-workload-socket\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.506644 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.506501    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-podinfo\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.506644 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.506557    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-credential-socket\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.506644 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.506602    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-workload-certs\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.506644 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.506626    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istiod-ca-cert\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.506816 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.506655    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-token\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.506816 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.506685    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmxmx\" (UniqueName: \"kubernetes.io/projected/637ed6a2-88e4-4c02-bc92-11cb39cadf43-kube-api-access-bmxmx\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.506816 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.506706    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-data\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.607823 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.607780    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmxmx\" (UniqueName: \"kubernetes.io/projected/637ed6a2-88e4-4c02-bc92-11cb39cadf43-kube-api-access-bmxmx\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.607823 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.607825    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-data\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.608346 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.607952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-envoy\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.608346 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.608032    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-workload-socket\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.608346 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.608053    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-podinfo\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.608346 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.608072    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-credential-socket\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.608346 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.608094    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-workload-certs\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.608346 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.608133    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istiod-ca-cert\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.608346 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.608168    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-token\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.608346 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.608244    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-data\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.608830 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.608510    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-credential-socket\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.608830 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.608608    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-workload-socket\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.608951 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.608873    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istiod-ca-cert\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.608951 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.608898    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-workload-certs\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.610269 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.610239    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-envoy\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.610464 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.610445    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-podinfo\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.616470 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.616413    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-token\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.616574 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.616501    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmxmx\" (UniqueName: \"kubernetes.io/projected/637ed6a2-88e4-4c02-bc92-11cb39cadf43-kube-api-access-bmxmx\") pod \"e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") " pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.681666 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.681622    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:31.818984 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.818941    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"]
Jul 20 20:36:31.820650 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:36:31.820624    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod637ed6a2_88e4_4c02_bc92_11cb39cadf43.slice/crio-b0d5bcbbcdabc9ac91193de7c476d758ea2d1d7283843370409a0a0deb18cefe WatchSource:0}: Error finding container b0d5bcbbcdabc9ac91193de7c476d758ea2d1d7283843370409a0a0deb18cefe: Status 404 returned error can't find the container with id b0d5bcbbcdabc9ac91193de7c476d758ea2d1d7283843370409a0a0deb18cefe
Jul 20 20:36:31.822932 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.822893    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:36:31.823025 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.822991    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:36:31.823064 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:31.823030    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:36:32.212089 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:32.211996    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn" event={"ID":"637ed6a2-88e4-4c02-bc92-11cb39cadf43","Type":"ContainerStarted","Data":"24db06b4ac5d88bc9e8b363a3b21274bc0831047e49c97d29dfa3124b55551bb"}
Jul 20 20:36:32.212089 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:32.212050    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn" event={"ID":"637ed6a2-88e4-4c02-bc92-11cb39cadf43","Type":"ContainerStarted","Data":"b0d5bcbbcdabc9ac91193de7c476d758ea2d1d7283843370409a0a0deb18cefe"}
Jul 20 20:36:32.231613 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:32.231566    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn" podStartSLOduration=1.231550024 podStartE2EDuration="1.231550024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:36:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:36:32.231029236 +0000 UTC m=+2407.495099282" watchObservedRunningTime="2026-07-20 20:36:32.231550024 +0000 UTC m=+2407.495620070"
Jul 20 20:36:32.682680 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:32.682639    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:32.687589 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:32.687566    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:33.216607 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:33.216581    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:33.217735 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:33.217714    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:35.760493 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:35.760469    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:36:35.950332 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:35.950297    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-workload-certs\") pod \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") "
Jul 20 20:36:35.950332 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:35.950341    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-workload-socket\") pod \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") "
Jul 20 20:36:35.950577 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:35.950361    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-podinfo\") pod \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") "
Jul 20 20:36:35.950577 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:35.950388    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpsx7\" (UniqueName: \"kubernetes.io/projected/6737b3e1-0fc8-4de0-9cd6-451b193d957c-kube-api-access-lpsx7\") pod \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") "
Jul 20 20:36:35.950577 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:35.950406    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-envoy\") pod \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") "
Jul 20 20:36:35.950577 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:35.950420    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-credential-socket\") pod \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") "
Jul 20 20:36:35.950577 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:35.950455    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istiod-ca-cert\") pod \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") "
Jul 20 20:36:35.950577 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:35.950497    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-data\") pod \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") "
Jul 20 20:36:35.950577 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:35.950549    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-token\") pod \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\" (UID: \"6737b3e1-0fc8-4de0-9cd6-451b193d957c\") "
Jul 20 20:36:35.950926 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:35.950583    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6737b3e1-0fc8-4de0-9cd6-451b193d957c" (UID: "6737b3e1-0fc8-4de0-9cd6-451b193d957c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:36:35.950926 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:35.950774    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6737b3e1-0fc8-4de0-9cd6-451b193d957c" (UID: "6737b3e1-0fc8-4de0-9cd6-451b193d957c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:36:35.950926 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:35.950799    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-workload-socket\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:35.950926 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:35.950877    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6737b3e1-0fc8-4de0-9cd6-451b193d957c" (UID: "6737b3e1-0fc8-4de0-9cd6-451b193d957c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 20:36:35.951145 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:35.950937    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6737b3e1-0fc8-4de0-9cd6-451b193d957c" (UID: "6737b3e1-0fc8-4de0-9cd6-451b193d957c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:36:35.951551 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:35.951524    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-data" (OuterVolumeSpecName: "istio-data") pod "6737b3e1-0fc8-4de0-9cd6-451b193d957c" (UID: "6737b3e1-0fc8-4de0-9cd6-451b193d957c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:36:35.952807 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:35.952785    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6737b3e1-0fc8-4de0-9cd6-451b193d957c" (UID: "6737b3e1-0fc8-4de0-9cd6-451b193d957c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 20:36:35.952807 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:35.952793    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6737b3e1-0fc8-4de0-9cd6-451b193d957c" (UID: "6737b3e1-0fc8-4de0-9cd6-451b193d957c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:36:35.953121 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:35.953100    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6737b3e1-0fc8-4de0-9cd6-451b193d957c-kube-api-access-lpsx7" (OuterVolumeSpecName: "kube-api-access-lpsx7") pod "6737b3e1-0fc8-4de0-9cd6-451b193d957c" (UID: "6737b3e1-0fc8-4de0-9cd6-451b193d957c"). InnerVolumeSpecName "kube-api-access-lpsx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:36:35.953121 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:35.953109    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-token" (OuterVolumeSpecName: "istio-token") pod "6737b3e1-0fc8-4de0-9cd6-451b193d957c" (UID: "6737b3e1-0fc8-4de0-9cd6-451b193d957c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:36:36.051749 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:36.051712    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istiod-ca-cert\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:36.051749 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:36.051742    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-data\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:36.051749 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:36.051751    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-token\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:36.051749 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:36.051759    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-workload-certs\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:36.052028 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:36.051766    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-podinfo\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:36.052028 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:36.051774    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpsx7\" (UniqueName: \"kubernetes.io/projected/6737b3e1-0fc8-4de0-9cd6-451b193d957c-kube-api-access-lpsx7\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:36.052028 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:36.051783    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-istio-envoy\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:36.052028 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:36.051791    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6737b3e1-0fc8-4de0-9cd6-451b193d957c-credential-socket\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:36.229922 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:36.229833    2580 generic.go:356] "Generic (PLEG): container finished" podID="6737b3e1-0fc8-4de0-9cd6-451b193d957c" containerID="32d0d62e6df2b3060d060802e5ba714a6fdc523c2e6dfb49604fd8a14cd41835" exitCode=0
Jul 20 20:36:36.229922 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:36.229881    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"
Jul 20 20:36:36.229922 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:36.229905    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm" event={"ID":"6737b3e1-0fc8-4de0-9cd6-451b193d957c","Type":"ContainerDied","Data":"32d0d62e6df2b3060d060802e5ba714a6fdc523c2e6dfb49604fd8a14cd41835"}
Jul 20 20:36:36.230165 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:36.229948    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm" event={"ID":"6737b3e1-0fc8-4de0-9cd6-451b193d957c","Type":"ContainerDied","Data":"84c758287603510836618e4604d915ab8984b94120bf5d8a237b0f47bccdb89d"}
Jul 20 20:36:36.230165 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:36.229988    2580 scope.go:117] "RemoveContainer" containerID="32d0d62e6df2b3060d060802e5ba714a6fdc523c2e6dfb49604fd8a14cd41835"
Jul 20 20:36:36.239462 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:36.239439    2580 scope.go:117] "RemoveContainer" containerID="c4d1d82ed74650067f3abd06c9e0e06dd6578b9f782e3ae5a88a09f1dacae7d3"
Jul 20 20:36:36.246785 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:36.246768    2580 scope.go:117] "RemoveContainer" containerID="32d0d62e6df2b3060d060802e5ba714a6fdc523c2e6dfb49604fd8a14cd41835"
Jul 20 20:36:36.247073 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:36:36.247048    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32d0d62e6df2b3060d060802e5ba714a6fdc523c2e6dfb49604fd8a14cd41835\": container with ID starting with 32d0d62e6df2b3060d060802e5ba714a6fdc523c2e6dfb49604fd8a14cd41835 not found: ID does not exist" containerID="32d0d62e6df2b3060d060802e5ba714a6fdc523c2e6dfb49604fd8a14cd41835"
Jul 20 20:36:36.247133 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:36.247082    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32d0d62e6df2b3060d060802e5ba714a6fdc523c2e6dfb49604fd8a14cd41835"} err="failed to get container status \"32d0d62e6df2b3060d060802e5ba714a6fdc523c2e6dfb49604fd8a14cd41835\": rpc error: code = NotFound desc = could not find container \"32d0d62e6df2b3060d060802e5ba714a6fdc523c2e6dfb49604fd8a14cd41835\": container with ID starting with 32d0d62e6df2b3060d060802e5ba714a6fdc523c2e6dfb49604fd8a14cd41835 not found: ID does not exist"
Jul 20 20:36:36.247133 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:36.247101    2580 scope.go:117] "RemoveContainer" containerID="c4d1d82ed74650067f3abd06c9e0e06dd6578b9f782e3ae5a88a09f1dacae7d3"
Jul 20 20:36:36.247311 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:36:36.247298    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4d1d82ed74650067f3abd06c9e0e06dd6578b9f782e3ae5a88a09f1dacae7d3\": container with ID starting with c4d1d82ed74650067f3abd06c9e0e06dd6578b9f782e3ae5a88a09f1dacae7d3 not found: ID does not exist" containerID="c4d1d82ed74650067f3abd06c9e0e06dd6578b9f782e3ae5a88a09f1dacae7d3"
Jul 20 20:36:36.247351 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:36.247314    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4d1d82ed74650067f3abd06c9e0e06dd6578b9f782e3ae5a88a09f1dacae7d3"} err="failed to get container status \"c4d1d82ed74650067f3abd06c9e0e06dd6578b9f782e3ae5a88a09f1dacae7d3\": rpc error: code = NotFound desc = could not find container \"c4d1d82ed74650067f3abd06c9e0e06dd6578b9f782e3ae5a88a09f1dacae7d3\": container with ID starting with c4d1d82ed74650067f3abd06c9e0e06dd6578b9f782e3ae5a88a09f1dacae7d3 not found: ID does not exist"
Jul 20 20:36:36.251560 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:36.251535    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"]
Jul 20 20:36:36.256183 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:36.256162    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-183471-openshift-default-68f765f47-xw6sm"]
Jul 20 20:36:37.175272 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:37.175240    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6737b3e1-0fc8-4de0-9cd6-451b193d957c" path="/var/lib/kubelet/pods/6737b3e1-0fc8-4de0-9cd6-451b193d957c/volumes"
Jul 20 20:36:49.242374 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:49.241655    2580 patch_prober.go:28] interesting pod/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 20 20:36:49.242374 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:49.241745    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn" podUID="637ed6a2-88e4-4c02-bc92-11cb39cadf43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 20 20:36:49.289897 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:49.289865    2580 generic.go:356] "Generic (PLEG): container finished" podID="637ed6a2-88e4-4c02-bc92-11cb39cadf43" containerID="24db06b4ac5d88bc9e8b363a3b21274bc0831047e49c97d29dfa3124b55551bb" exitCode=0
Jul 20 20:36:49.290068 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:49.289949    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn" event={"ID":"637ed6a2-88e4-4c02-bc92-11cb39cadf43","Type":"ContainerDied","Data":"24db06b4ac5d88bc9e8b363a3b21274bc0831047e49c97d29dfa3124b55551bb"}
Jul 20 20:36:49.290363 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:49.290345    2580 scope.go:117] "RemoveContainer" containerID="24db06b4ac5d88bc9e8b363a3b21274bc0831047e49c97d29dfa3124b55551bb"
Jul 20 20:36:49.291303 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:49.291277    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:36:49.291354 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:49.291345    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:36:49.291389 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:49.291377    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:36:50.296672 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:50.296639    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn" event={"ID":"637ed6a2-88e4-4c02-bc92-11cb39cadf43","Type":"ContainerStarted","Data":"edf3146d829f2271bf5833c2e20ae2731aec0f6f969e0761164719da0a197597"}
Jul 20 20:36:50.682077 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:50.682034    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:50.682311 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:50.682169    2580 patch_prober.go:28] interesting pod/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 20 20:36:50.682311 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:50.682229    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn" podUID="637ed6a2-88e4-4c02-bc92-11cb39cadf43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 20 20:36:51.683060 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:51.683013    2580 patch_prober.go:28] interesting pod/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 20 20:36:51.683571 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:51.683085    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn" podUID="637ed6a2-88e4-4c02-bc92-11cb39cadf43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 20 20:36:52.682259 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:52.682193    2580 patch_prober.go:28] interesting pod/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 20 20:36:52.682488 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:52.682292    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn" podUID="637ed6a2-88e4-4c02-bc92-11cb39cadf43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 20 20:36:53.682871 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:53.682824    2580 patch_prober.go:28] interesting pod/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 20 20:36:53.683384 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:53.682904    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn" podUID="637ed6a2-88e4-4c02-bc92-11cb39cadf43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 20 20:36:54.686460 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:54.686427    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:54.686827 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:54.686791    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:54.687414 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:54.687392    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:36:55.363156 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:55.363119    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93c18a-6nw5d"]
Jul 20 20:36:55.363441 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:55.363427    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6737b3e1-0fc8-4de0-9cd6-451b193d957c" containerName="istio-proxy"
Jul 20 20:36:55.363495 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:55.363444    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6737b3e1-0fc8-4de0-9cd6-451b193d957c" containerName="istio-proxy"
Jul 20 20:36:55.363531 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:55.363506    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6737b3e1-0fc8-4de0-9cd6-451b193d957c" containerName="istio-proxy"
Jul 20 20:36:55.363531 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:55.363519    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6737b3e1-0fc8-4de0-9cd6-451b193d957c" containerName="istio-proxy"
Jul 20 20:36:55.366664 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:55.366641    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93c18a-6nw5d"
Jul 20 20:36:55.369608 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:55.369585    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-628hb\""
Jul 20 20:36:55.369750 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:55.369586    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 20:36:55.373940 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:55.373910    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93c18a-6nw5d"]
Jul 20 20:36:55.417317 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:55.417278    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-93c18a-6nw5d\" (UID: \"1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93c18a-6nw5d"
Jul 20 20:36:55.518226 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:55.518193    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-93c18a-6nw5d\" (UID: \"1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93c18a-6nw5d"
Jul 20 20:36:55.518821 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:55.518800    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-93c18a-6nw5d\" (UID: \"1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93c18a-6nw5d"
Jul 20 20:36:55.680338 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:55.680261    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93c18a-6nw5d"
Jul 20 20:36:55.815025 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:55.814981    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93c18a-6nw5d"]
Jul 20 20:36:55.817799 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:36:55.817774    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e4649e2_73f4_4c4b_88fc_1e6bbb7a299a.slice/crio-776498a9c519850dca86050e279992bc3a380ee2e729130f42e29a27752cdbbd WatchSource:0}: Error finding container 776498a9c519850dca86050e279992bc3a380ee2e729130f42e29a27752cdbbd: Status 404 returned error can't find the container with id 776498a9c519850dca86050e279992bc3a380ee2e729130f42e29a27752cdbbd
Jul 20 20:36:56.324561 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:56.324521    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93c18a-6nw5d" event={"ID":"1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a","Type":"ContainerStarted","Data":"5f62176ea66bf325cc5462733da1b710c644e216c7ec62fea4ea9d2c189ee179"}
Jul 20 20:36:56.324749 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:56.324583    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93c18a-6nw5d" event={"ID":"1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a","Type":"ContainerStarted","Data":"776498a9c519850dca86050e279992bc3a380ee2e729130f42e29a27752cdbbd"}
Jul 20 20:36:56.339920 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:56.339874    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93c18a-6nw5d" podStartSLOduration=1.339859367 podStartE2EDuration="1.339859367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:36:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:36:56.339243694 +0000 UTC m=+2431.603313743" watchObservedRunningTime="2026-07-20 20:36:56.339859367 +0000 UTC m=+2431.603929413"
Jul 20 20:36:57.330527 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:57.330488    2580 generic.go:356] "Generic (PLEG): container finished" podID="1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a" containerID="5f62176ea66bf325cc5462733da1b710c644e216c7ec62fea4ea9d2c189ee179" exitCode=0
Jul 20 20:36:57.330909 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:57.330574    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93c18a-6nw5d" event={"ID":"1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a","Type":"ContainerDied","Data":"5f62176ea66bf325cc5462733da1b710c644e216c7ec62fea4ea9d2c189ee179"}
Jul 20 20:36:58.464441 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:58.464413    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93c18a-6nw5d"
Jul 20 20:36:58.544192 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:58.544164    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a-maas-api-service-ca\") pod \"1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a\" (UID: \"1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a\") "
Jul 20 20:36:58.544520 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:58.544499    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a" (UID: "1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 20:36:58.645685 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:58.645605    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a-maas-api-service-ca\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:36:59.342254 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:59.342220    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93c18a-6nw5d" event={"ID":"1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a","Type":"ContainerDied","Data":"776498a9c519850dca86050e279992bc3a380ee2e729130f42e29a27752cdbbd"}
Jul 20 20:36:59.342429 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:59.342271    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="776498a9c519850dca86050e279992bc3a380ee2e729130f42e29a27752cdbbd"
Jul 20 20:36:59.342429 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:36:59.342231    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93c18a-6nw5d"
Jul 20 20:37:15.437170 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:15.437135    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93c18a-6nw5d"]
Jul 20 20:37:15.439760 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:15.439734    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93c18a-6nw5d"]
Jul 20 20:37:17.175494 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:17.175459    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a" path="/var/lib/kubelet/pods/1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a/volumes"
Jul 20 20:37:22.446893 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:22.446862    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"]
Jul 20 20:37:22.447385 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:22.447141    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn" podUID="637ed6a2-88e4-4c02-bc92-11cb39cadf43" containerName="istio-proxy" containerID="cri-o://edf3146d829f2271bf5833c2e20ae2731aec0f6f969e0761164719da0a197597" gracePeriod=30
Jul 20 20:37:22.810721 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:22.810681    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f9c27f-ghp4m"]
Jul 20 20:37:22.811087 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:22.811069    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a" containerName="revoke-keys"
Jul 20 20:37:22.811192 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:22.811089    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a" containerName="revoke-keys"
Jul 20 20:37:22.811192 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:22.811110    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6737b3e1-0fc8-4de0-9cd6-451b193d957c" containerName="istio-proxy"
Jul 20 20:37:22.811192 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:22.811118    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6737b3e1-0fc8-4de0-9cd6-451b193d957c" containerName="istio-proxy"
Jul 20 20:37:22.811351 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:22.811206    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e4649e2-73f4-4c4b-88fc-1e6bbb7a299a" containerName="revoke-keys"
Jul 20 20:37:22.814186 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:22.814166    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f9c27f-ghp4m"
Jul 20 20:37:22.817096 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:22.817075    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 20:37:22.817188 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:22.817075    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-628hb\""
Jul 20 20:37:22.822271 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:22.822246    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f9c27f-ghp4m"]
Jul 20 20:37:22.960393 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:22.960363    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/287283b1-53b3-4aae-9856-303ae642af37-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f9c27f-ghp4m\" (UID: \"287283b1-53b3-4aae-9856-303ae642af37\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f9c27f-ghp4m"
Jul 20 20:37:23.061099 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:23.061009    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/287283b1-53b3-4aae-9856-303ae642af37-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f9c27f-ghp4m\" (UID: \"287283b1-53b3-4aae-9856-303ae642af37\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f9c27f-ghp4m"
Jul 20 20:37:23.061873 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:23.061850    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/287283b1-53b3-4aae-9856-303ae642af37-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f9c27f-ghp4m\" (UID: \"287283b1-53b3-4aae-9856-303ae642af37\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f9c27f-ghp4m"
Jul 20 20:37:23.124690 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:23.124661    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f9c27f-ghp4m"
Jul 20 20:37:23.271367 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:23.271238    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f9c27f-ghp4m"]
Jul 20 20:37:23.273852 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:37:23.273828    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod287283b1_53b3_4aae_9856_303ae642af37.slice/crio-3f56f49af27df1041bbceb23dbb1ad744fdd0eac5106e6ecc869a72fdde446b0 WatchSource:0}: Error finding container 3f56f49af27df1041bbceb23dbb1ad744fdd0eac5106e6ecc869a72fdde446b0: Status 404 returned error can't find the container with id 3f56f49af27df1041bbceb23dbb1ad744fdd0eac5106e6ecc869a72fdde446b0
Jul 20 20:37:23.450208 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:23.450121    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f9c27f-ghp4m" event={"ID":"287283b1-53b3-4aae-9856-303ae642af37","Type":"ContainerStarted","Data":"3f56f49af27df1041bbceb23dbb1ad744fdd0eac5106e6ecc869a72fdde446b0"}
Jul 20 20:37:24.456147 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:24.456060    2580 generic.go:356] "Generic (PLEG): container finished" podID="287283b1-53b3-4aae-9856-303ae642af37" containerID="941a677245e96f1a68c742bd19d616696ef8c113731ccfbeaa8e410451f8f5d6" exitCode=0
Jul 20 20:37:24.456147 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:24.456105    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f9c27f-ghp4m" event={"ID":"287283b1-53b3-4aae-9856-303ae642af37","Type":"ContainerDied","Data":"941a677245e96f1a68c742bd19d616696ef8c113731ccfbeaa8e410451f8f5d6"}
Jul 20 20:37:24.689399 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:24.689363    2580 patch_prober.go:28] interesting pod/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 20 20:37:24.689581 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:24.689420    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn" podUID="637ed6a2-88e4-4c02-bc92-11cb39cadf43" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 20 20:37:25.385568 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:25.385532    2580 scope.go:117] "RemoveContainer" containerID="490679b2b5cd0124620843970a6d7bca5dfc15222b7551da15b9facd5865581b"
Jul 20 20:37:25.587833 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:25.587806    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f9c27f-ghp4m"
Jul 20 20:37:25.685201 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:25.685123    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/287283b1-53b3-4aae-9856-303ae642af37-maas-api-service-ca\") pod \"287283b1-53b3-4aae-9856-303ae642af37\" (UID: \"287283b1-53b3-4aae-9856-303ae642af37\") "
Jul 20 20:37:25.685484 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:25.685464    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/287283b1-53b3-4aae-9856-303ae642af37-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "287283b1-53b3-4aae-9856-303ae642af37" (UID: "287283b1-53b3-4aae-9856-303ae642af37"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 20:37:25.786196 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:25.786161    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/287283b1-53b3-4aae-9856-303ae642af37-maas-api-service-ca\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:37:26.466163 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:26.466136    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f9c27f-ghp4m"
Jul 20 20:37:26.466163 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:26.466156    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f9c27f-ghp4m" event={"ID":"287283b1-53b3-4aae-9856-303ae642af37","Type":"ContainerDied","Data":"3f56f49af27df1041bbceb23dbb1ad744fdd0eac5106e6ecc869a72fdde446b0"}
Jul 20 20:37:26.466367 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:26.466201    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f56f49af27df1041bbceb23dbb1ad744fdd0eac5106e6ecc869a72fdde446b0"
Jul 20 20:37:27.697547 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.697522    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:37:27.804926 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.804893    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-workload-socket\") pod \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") "
Jul 20 20:37:27.805113 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.804937    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-workload-certs\") pod \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") "
Jul 20 20:37:27.805113 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.804983    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-token\") pod \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") "
Jul 20 20:37:27.805113 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.805013    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmxmx\" (UniqueName: \"kubernetes.io/projected/637ed6a2-88e4-4c02-bc92-11cb39cadf43-kube-api-access-bmxmx\") pod \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") "
Jul 20 20:37:27.805113 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.805056    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istiod-ca-cert\") pod \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") "
Jul 20 20:37:27.805300 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.805130    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-data\") pod \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") "
Jul 20 20:37:27.805300 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.805184    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-podinfo\") pod \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") "
Jul 20 20:37:27.805300 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.805207    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-credential-socket\") pod \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") "
Jul 20 20:37:27.805300 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.805237    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-envoy\") pod \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\" (UID: \"637ed6a2-88e4-4c02-bc92-11cb39cadf43\") "
Jul 20 20:37:27.805506 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.805236    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "637ed6a2-88e4-4c02-bc92-11cb39cadf43" (UID: "637ed6a2-88e4-4c02-bc92-11cb39cadf43"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:37:27.805506 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.805451    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "637ed6a2-88e4-4c02-bc92-11cb39cadf43" (UID: "637ed6a2-88e4-4c02-bc92-11cb39cadf43"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:37:27.805618 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.805587    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "637ed6a2-88e4-4c02-bc92-11cb39cadf43" (UID: "637ed6a2-88e4-4c02-bc92-11cb39cadf43"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 20:37:27.805618 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.805606    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "637ed6a2-88e4-4c02-bc92-11cb39cadf43" (UID: "637ed6a2-88e4-4c02-bc92-11cb39cadf43"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:37:27.806065 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.806041    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-data" (OuterVolumeSpecName: "istio-data") pod "637ed6a2-88e4-4c02-bc92-11cb39cadf43" (UID: "637ed6a2-88e4-4c02-bc92-11cb39cadf43"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:37:27.807421 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.807396    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "637ed6a2-88e4-4c02-bc92-11cb39cadf43" (UID: "637ed6a2-88e4-4c02-bc92-11cb39cadf43"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:37:27.807528 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.807443    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "637ed6a2-88e4-4c02-bc92-11cb39cadf43" (UID: "637ed6a2-88e4-4c02-bc92-11cb39cadf43"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 20:37:27.807674 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.807656    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/637ed6a2-88e4-4c02-bc92-11cb39cadf43-kube-api-access-bmxmx" (OuterVolumeSpecName: "kube-api-access-bmxmx") pod "637ed6a2-88e4-4c02-bc92-11cb39cadf43" (UID: "637ed6a2-88e4-4c02-bc92-11cb39cadf43"). InnerVolumeSpecName "kube-api-access-bmxmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:37:27.807734 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.807657    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-token" (OuterVolumeSpecName: "istio-token") pod "637ed6a2-88e4-4c02-bc92-11cb39cadf43" (UID: "637ed6a2-88e4-4c02-bc92-11cb39cadf43"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:37:27.906863 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.906831    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-podinfo\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:37:27.906863 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.906858    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-credential-socket\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:37:27.906863 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.906869    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-envoy\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:37:27.907125 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.906877    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-workload-socket\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:37:27.907125 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.906886    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-workload-certs\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:37:27.907125 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.906894    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-token\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:37:27.907125 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.906902    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmxmx\" (UniqueName: \"kubernetes.io/projected/637ed6a2-88e4-4c02-bc92-11cb39cadf43-kube-api-access-bmxmx\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:37:27.907125 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.906912    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istiod-ca-cert\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:37:27.907125 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:27.906919    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/637ed6a2-88e4-4c02-bc92-11cb39cadf43-istio-data\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:37:28.477729 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:28.477698    2580 generic.go:356] "Generic (PLEG): container finished" podID="637ed6a2-88e4-4c02-bc92-11cb39cadf43" containerID="edf3146d829f2271bf5833c2e20ae2731aec0f6f969e0761164719da0a197597" exitCode=0
Jul 20 20:37:28.477940 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:28.477756    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"
Jul 20 20:37:28.477940 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:28.477778    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn" event={"ID":"637ed6a2-88e4-4c02-bc92-11cb39cadf43","Type":"ContainerDied","Data":"edf3146d829f2271bf5833c2e20ae2731aec0f6f969e0761164719da0a197597"}
Jul 20 20:37:28.477940 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:28.477832    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn" event={"ID":"637ed6a2-88e4-4c02-bc92-11cb39cadf43","Type":"ContainerDied","Data":"b0d5bcbbcdabc9ac91193de7c476d758ea2d1d7283843370409a0a0deb18cefe"}
Jul 20 20:37:28.477940 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:28.477857    2580 scope.go:117] "RemoveContainer" containerID="edf3146d829f2271bf5833c2e20ae2731aec0f6f969e0761164719da0a197597"
Jul 20 20:37:28.487051 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:28.487035    2580 scope.go:117] "RemoveContainer" containerID="24db06b4ac5d88bc9e8b363a3b21274bc0831047e49c97d29dfa3124b55551bb"
Jul 20 20:37:28.494761 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:28.494745    2580 scope.go:117] "RemoveContainer" containerID="edf3146d829f2271bf5833c2e20ae2731aec0f6f969e0761164719da0a197597"
Jul 20 20:37:28.495047 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:37:28.495027    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edf3146d829f2271bf5833c2e20ae2731aec0f6f969e0761164719da0a197597\": container with ID starting with edf3146d829f2271bf5833c2e20ae2731aec0f6f969e0761164719da0a197597 not found: ID does not exist" containerID="edf3146d829f2271bf5833c2e20ae2731aec0f6f969e0761164719da0a197597"
Jul 20 20:37:28.495125 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:28.495056    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edf3146d829f2271bf5833c2e20ae2731aec0f6f969e0761164719da0a197597"} err="failed to get container status \"edf3146d829f2271bf5833c2e20ae2731aec0f6f969e0761164719da0a197597\": rpc error: code = NotFound desc = could not find container \"edf3146d829f2271bf5833c2e20ae2731aec0f6f969e0761164719da0a197597\": container with ID starting with edf3146d829f2271bf5833c2e20ae2731aec0f6f969e0761164719da0a197597 not found: ID does not exist"
Jul 20 20:37:28.495125 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:28.495074    2580 scope.go:117] "RemoveContainer" containerID="24db06b4ac5d88bc9e8b363a3b21274bc0831047e49c97d29dfa3124b55551bb"
Jul 20 20:37:28.495302 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:37:28.495283    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24db06b4ac5d88bc9e8b363a3b21274bc0831047e49c97d29dfa3124b55551bb\": container with ID starting with 24db06b4ac5d88bc9e8b363a3b21274bc0831047e49c97d29dfa3124b55551bb not found: ID does not exist" containerID="24db06b4ac5d88bc9e8b363a3b21274bc0831047e49c97d29dfa3124b55551bb"
Jul 20 20:37:28.495362 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:28.495313    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24db06b4ac5d88bc9e8b363a3b21274bc0831047e49c97d29dfa3124b55551bb"} err="failed to get container status \"24db06b4ac5d88bc9e8b363a3b21274bc0831047e49c97d29dfa3124b55551bb\": rpc error: code = NotFound desc = could not find container \"24db06b4ac5d88bc9e8b363a3b21274bc0831047e49c97d29dfa3124b55551bb\": container with ID starting with 24db06b4ac5d88bc9e8b363a3b21274bc0831047e49c97d29dfa3124b55551bb not found: ID does not exist"
Jul 20 20:37:28.500391 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:28.500372    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"]
Jul 20 20:37:28.504754 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:28.504734    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-93c18a-openshift-default-64d8bd569-j55pn"]
Jul 20 20:37:29.176885 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:29.176854    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="637ed6a2-88e4-4c02-bc92-11cb39cadf43" path="/var/lib/kubelet/pods/637ed6a2-88e4-4c02-bc92-11cb39cadf43/volumes"
Jul 20 20:37:42.891466 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:42.891431    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f9c27f-ghp4m"]
Jul 20 20:37:42.893520 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:42.893495    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f9c27f-ghp4m"]
Jul 20 20:37:43.174950 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:43.174863    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="287283b1-53b3-4aae-9856-303ae642af37" path="/var/lib/kubelet/pods/287283b1-53b3-4aae-9856-303ae642af37/volumes"
Jul 20 20:37:50.490828 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.490776    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"]
Jul 20 20:37:50.491295 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.491208    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="287283b1-53b3-4aae-9856-303ae642af37" containerName="revoke-keys"
Jul 20 20:37:50.491295 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.491222    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="287283b1-53b3-4aae-9856-303ae642af37" containerName="revoke-keys"
Jul 20 20:37:50.491295 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.491241    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="637ed6a2-88e4-4c02-bc92-11cb39cadf43" containerName="istio-proxy"
Jul 20 20:37:50.491295 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.491250    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="637ed6a2-88e4-4c02-bc92-11cb39cadf43" containerName="istio-proxy"
Jul 20 20:37:50.491295 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.491267    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="637ed6a2-88e4-4c02-bc92-11cb39cadf43" containerName="istio-proxy"
Jul 20 20:37:50.491295 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.491276    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="637ed6a2-88e4-4c02-bc92-11cb39cadf43" containerName="istio-proxy"
Jul 20 20:37:50.491482 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.491345    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="637ed6a2-88e4-4c02-bc92-11cb39cadf43" containerName="istio-proxy"
Jul 20 20:37:50.491482 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.491355    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="287283b1-53b3-4aae-9856-303ae642af37" containerName="revoke-keys"
Jul 20 20:37:50.491482 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.491365    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="637ed6a2-88e4-4c02-bc92-11cb39cadf43" containerName="istio-proxy"
Jul 20 20:37:50.494388 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.494367    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.497197 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.497173    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 20 20:37:50.497440 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.497424    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-b15c6d-openshift-default-dockercfg-v7qtj\""
Jul 20 20:37:50.512515 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.512486    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"]
Jul 20 20:37:50.580564 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.580531    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-workload-socket\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.580755 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.580581    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f938e0df-8655-4de8-940d-6bcf9b227275-istio-podinfo\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.580755 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.580638    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpxdx\" (UniqueName: \"kubernetes.io/projected/f938e0df-8655-4de8-940d-6bcf9b227275-kube-api-access-zpxdx\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.580755 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.580670    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-credential-socket\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.580755 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.580702    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-istio-envoy\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.580755 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.580718    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-istio-data\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.581067 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.580757    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f938e0df-8655-4de8-940d-6bcf9b227275-istiod-ca-cert\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.581067 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.580780    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-workload-certs\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.581067 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.580811    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f938e0df-8655-4de8-940d-6bcf9b227275-istio-token\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.681707 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.681671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f938e0df-8655-4de8-940d-6bcf9b227275-istio-token\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.681707 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.681714    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-workload-socket\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.681916 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.681829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f938e0df-8655-4de8-940d-6bcf9b227275-istio-podinfo\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.681916 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.681892    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpxdx\" (UniqueName: \"kubernetes.io/projected/f938e0df-8655-4de8-940d-6bcf9b227275-kube-api-access-zpxdx\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.682032 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.681927    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-credential-socket\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.682032 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.681997    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-istio-envoy\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.682032 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.682021    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-istio-data\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.682197 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.682045    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f938e0df-8655-4de8-940d-6bcf9b227275-istiod-ca-cert\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.682197 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.682079    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-workload-certs\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.682197 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.682103    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-workload-socket\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.682358 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.682310    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-credential-socket\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.682461 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.682436    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-istio-data\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.682604 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.682488    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-workload-certs\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.682742 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.682723    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f938e0df-8655-4de8-940d-6bcf9b227275-istiod-ca-cert\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.684269 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.684245    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-istio-envoy\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.684351 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.684290    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f938e0df-8655-4de8-940d-6bcf9b227275-istio-podinfo\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.690066 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.690044    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f938e0df-8655-4de8-940d-6bcf9b227275-istio-token\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.690555 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.690535    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpxdx\" (UniqueName: \"kubernetes.io/projected/f938e0df-8655-4de8-940d-6bcf9b227275-kube-api-access-zpxdx\") pod \"e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") " pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.806174 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.806132    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:50.940070 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.940043    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"]
Jul 20 20:37:50.942493 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:37:50.942467    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf938e0df_8655_4de8_940d_6bcf9b227275.slice/crio-5f2b98837d79639950152c00bb4cfbe32beb6badefd8da0b0c447a51083ee6c9 WatchSource:0}: Error finding container 5f2b98837d79639950152c00bb4cfbe32beb6badefd8da0b0c447a51083ee6c9: Status 404 returned error can't find the container with id 5f2b98837d79639950152c00bb4cfbe32beb6badefd8da0b0c447a51083ee6c9
Jul 20 20:37:50.944267 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.944230    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:37:50.944359 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.944297    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:37:50.944359 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:50.944331    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:37:51.587138 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:51.587095    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67" event={"ID":"f938e0df-8655-4de8-940d-6bcf9b227275","Type":"ContainerStarted","Data":"f47f279000f507b6c264ae50ae91f5c11680b9b61ecc80c5a97d0fb9a2630485"}
Jul 20 20:37:51.587622 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:51.587183    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67" event={"ID":"f938e0df-8655-4de8-940d-6bcf9b227275","Type":"ContainerStarted","Data":"5f2b98837d79639950152c00bb4cfbe32beb6badefd8da0b0c447a51083ee6c9"}
Jul 20 20:37:51.615769 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:51.615717    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67" podStartSLOduration=1.6157034810000002 podStartE2EDuration="1.615703481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:37:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:37:51.614083963 +0000 UTC m=+2486.878154013" watchObservedRunningTime="2026-07-20 20:37:51.615703481 +0000 UTC m=+2486.879773527"
Jul 20 20:37:51.806663 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:51.806635    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:51.811330 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:51.811305    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:52.591474 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:52.591447    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:37:52.592368 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:37:52.592347    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:38:08.634113 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:08.634076    2580 patch_prober.go:28] interesting pod/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 20 20:38:08.634506 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:08.634154    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67" podUID="f938e0df-8655-4de8-940d-6bcf9b227275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 20 20:38:09.668968 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:09.668922    2580 generic.go:356] "Generic (PLEG): container finished" podID="f938e0df-8655-4de8-940d-6bcf9b227275" containerID="f47f279000f507b6c264ae50ae91f5c11680b9b61ecc80c5a97d0fb9a2630485" exitCode=0
Jul 20 20:38:09.669383 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:09.668996    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67" event={"ID":"f938e0df-8655-4de8-940d-6bcf9b227275","Type":"ContainerDied","Data":"f47f279000f507b6c264ae50ae91f5c11680b9b61ecc80c5a97d0fb9a2630485"}
Jul 20 20:38:09.669383 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:09.669346    2580 scope.go:117] "RemoveContainer" containerID="f47f279000f507b6c264ae50ae91f5c11680b9b61ecc80c5a97d0fb9a2630485"
Jul 20 20:38:09.670324 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:09.670299    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:38:09.670377 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:09.670366    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:38:09.670424 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:09.670412    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 20:38:10.678545 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:10.678503    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67" event={"ID":"f938e0df-8655-4de8-940d-6bcf9b227275","Type":"ContainerStarted","Data":"04e2cd3d144fae9940cf949aca86d05a9fc95ddcd6c6ff17505939676948cac7"}
Jul 20 20:38:10.806569 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:10.806534    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:38:10.806759 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:10.806740    2580 patch_prober.go:28] interesting pod/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 20 20:38:10.806804 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:10.806784    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67" podUID="f938e0df-8655-4de8-940d-6bcf9b227275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 20 20:38:11.806845 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:11.806807    2580 patch_prober.go:28] interesting pod/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 20 20:38:11.807280 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:11.806864    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67" podUID="f938e0df-8655-4de8-940d-6bcf9b227275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 20 20:38:12.807273 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:12.807231    2580 patch_prober.go:28] interesting pod/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 20 20:38:12.807660 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:12.807288    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67" podUID="f938e0df-8655-4de8-940d-6bcf9b227275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 20 20:38:13.806681 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:13.806641    2580 patch_prober.go:28] interesting pod/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 20 20:38:13.806859 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:13.806702    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67" podUID="f938e0df-8655-4de8-940d-6bcf9b227275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 20 20:38:14.809974 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:14.809930    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:38:14.810387 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:14.810270    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:38:14.810844 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:14.810828    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:38:43.212712 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.212678    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"]
Jul 20 20:38:43.215824 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.215804    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:43.218456 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.218434    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b15c6d\"/\"kube-root-ca.crt\""
Jul 20 20:38:43.219869 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.219841    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b15c6d\"/\"auth-test-model-b15c6d-kserve-self-signed-certs\""
Jul 20 20:38:43.219869 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.219864    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b15c6d\"/\"openshift-service-ca.crt\""
Jul 20 20:38:43.220070 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.219846    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b15c6d\"/\"default-dockercfg-6v24x\""
Jul 20 20:38:43.226400 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.226377    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"]
Jul 20 20:38:43.247632 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.247595    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-home\") pod \"auth-test-model-b15c6d-kserve-658cc49bdc-2w56f\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") " pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:43.247778 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.247650    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-tmp-dir\") pod \"auth-test-model-b15c6d-kserve-658cc49bdc-2w56f\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") " pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:43.247778 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.247680    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-dshm\") pod \"auth-test-model-b15c6d-kserve-658cc49bdc-2w56f\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") " pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:43.247778 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.247728    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-tls-certs\") pod \"auth-test-model-b15c6d-kserve-658cc49bdc-2w56f\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") " pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:43.247888 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.247778    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-model-cache\") pod \"auth-test-model-b15c6d-kserve-658cc49bdc-2w56f\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") " pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:43.247888 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.247827    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxnln\" (UniqueName: \"kubernetes.io/projected/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-kube-api-access-pxnln\") pod \"auth-test-model-b15c6d-kserve-658cc49bdc-2w56f\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") " pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:43.348902 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.348862    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-home\") pod \"auth-test-model-b15c6d-kserve-658cc49bdc-2w56f\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") " pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:43.348902 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.348903    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-tmp-dir\") pod \"auth-test-model-b15c6d-kserve-658cc49bdc-2w56f\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") " pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:43.349201 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.348927    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-dshm\") pod \"auth-test-model-b15c6d-kserve-658cc49bdc-2w56f\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") " pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:43.349201 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.348980    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-tls-certs\") pod \"auth-test-model-b15c6d-kserve-658cc49bdc-2w56f\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") " pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:43.349201 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.348999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-model-cache\") pod \"auth-test-model-b15c6d-kserve-658cc49bdc-2w56f\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") " pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:43.349201 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.349147    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxnln\" (UniqueName: \"kubernetes.io/projected/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-kube-api-access-pxnln\") pod \"auth-test-model-b15c6d-kserve-658cc49bdc-2w56f\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") " pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:43.349419 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.349333    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-tmp-dir\") pod \"auth-test-model-b15c6d-kserve-658cc49bdc-2w56f\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") " pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:43.349419 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.349391    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-model-cache\") pod \"auth-test-model-b15c6d-kserve-658cc49bdc-2w56f\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") " pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:43.349505 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.349435    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-home\") pod \"auth-test-model-b15c6d-kserve-658cc49bdc-2w56f\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") " pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:43.351344 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.351321    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-dshm\") pod \"auth-test-model-b15c6d-kserve-658cc49bdc-2w56f\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") " pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:43.351543 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.351527    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-tls-certs\") pod \"auth-test-model-b15c6d-kserve-658cc49bdc-2w56f\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") " pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:43.357434 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.357411    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxnln\" (UniqueName: \"kubernetes.io/projected/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-kube-api-access-pxnln\") pod \"auth-test-model-b15c6d-kserve-658cc49bdc-2w56f\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") " pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:43.524533 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.524452    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:43.653506 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.653466    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"]
Jul 20 20:38:43.657490 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:38:43.657465    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb71dbc0_0fd2_43df_9abd_8e63a2b28e80.slice/crio-cb9d0dba35e5a35812bfc558db05c6100d2ce91a033170d89b81ee3578829ccd WatchSource:0}: Error finding container cb9d0dba35e5a35812bfc558db05c6100d2ce91a033170d89b81ee3578829ccd: Status 404 returned error can't find the container with id cb9d0dba35e5a35812bfc558db05c6100d2ce91a033170d89b81ee3578829ccd
Jul 20 20:38:43.832860 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.832828    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f" event={"ID":"db71dbc0-0fd2-43df-9abd-8e63a2b28e80","Type":"ContainerStarted","Data":"0ef61af3645a7df6f985804477e32a0d2b89a151ff66ab163683e48d70502053"}
Jul 20 20:38:43.833069 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.832870    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f" event={"ID":"db71dbc0-0fd2-43df-9abd-8e63a2b28e80","Type":"ContainerStarted","Data":"cb9d0dba35e5a35812bfc558db05c6100d2ce91a033170d89b81ee3578829ccd"}
Jul 20 20:38:43.849900 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:43.849849    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f" podStartSLOduration=0.849835832 podStartE2EDuration="849.835832ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:38:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:38:43.848894296 +0000 UTC m=+2539.112964353" watchObservedRunningTime="2026-07-20 20:38:43.849835832 +0000 UTC m=+2539.113905878"
Jul 20 20:38:53.524854 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:53.524819    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:53.525322 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:53.524869    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:53.537177 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:53.537152    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:38:53.886973 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:38:53.886928    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:39:03.918270 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:03.918238    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"]
Jul 20 20:39:03.921215 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:03.921198    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:03.924191 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:03.924167    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-40065f\"/\"openshift-service-ca.crt\""
Jul 20 20:39:03.925725 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:03.925706    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-40065f\"/\"kube-root-ca.crt\""
Jul 20 20:39:03.925841 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:03.925813    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-40065f\"/\"auth-test-model-40065f-kserve-self-signed-certs\""
Jul 20 20:39:03.925892 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:03.925824    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-40065f\"/\"default-dockercfg-84tz8\""
Jul 20 20:39:03.930421 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:03.930400    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"]
Jul 20 20:39:04.025413 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.025362    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-tmp-dir\") pod \"auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") " pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:04.025413 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.025424    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-home\") pod \"auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") " pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:04.025640 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.025450    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-model-cache\") pod \"auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") " pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:04.025640 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.025484    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bffab009-93ae-428b-bf97-790fd61322a1-tls-certs\") pod \"auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") " pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:04.025640 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.025513    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-dshm\") pod \"auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") " pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:04.025640 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.025546    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-846jd\" (UniqueName: \"kubernetes.io/projected/bffab009-93ae-428b-bf97-790fd61322a1-kube-api-access-846jd\") pod \"auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") " pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:04.126383 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.126348    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-tmp-dir\") pod \"auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") " pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:04.126589 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.126406    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-home\") pod \"auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") " pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:04.126589 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.126514    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-model-cache\") pod \"auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") " pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:04.126589 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.126558    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bffab009-93ae-428b-bf97-790fd61322a1-tls-certs\") pod \"auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") " pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:04.126756 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.126617    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-dshm\") pod \"auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") " pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:04.126756 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.126653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-846jd\" (UniqueName: \"kubernetes.io/projected/bffab009-93ae-428b-bf97-790fd61322a1-kube-api-access-846jd\") pod \"auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") " pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:04.126881 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.126812    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-tmp-dir\") pod \"auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") " pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:04.126933 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.126885    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-home\") pod \"auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") " pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:04.126933 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.126900    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-model-cache\") pod \"auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") " pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:04.128969 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.128938    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-dshm\") pod \"auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") " pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:04.129108 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.129093    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bffab009-93ae-428b-bf97-790fd61322a1-tls-certs\") pod \"auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") " pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:04.135939 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.135912    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-846jd\" (UniqueName: \"kubernetes.io/projected/bffab009-93ae-428b-bf97-790fd61322a1-kube-api-access-846jd\") pod \"auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") " pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:04.232241 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.232144    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:04.367534 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.367500    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"]
Jul 20 20:39:04.370736 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:39:04.370708    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbffab009_93ae_428b_bf97_790fd61322a1.slice/crio-91d406d4d0e08225eab036d672b4ad68b66d45c8189002b76d2e3ea611cc45ac WatchSource:0}: Error finding container 91d406d4d0e08225eab036d672b4ad68b66d45c8189002b76d2e3ea611cc45ac: Status 404 returned error can't find the container with id 91d406d4d0e08225eab036d672b4ad68b66d45c8189002b76d2e3ea611cc45ac
Jul 20 20:39:04.925760 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.925722    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw" event={"ID":"bffab009-93ae-428b-bf97-790fd61322a1","Type":"ContainerStarted","Data":"b50ba928470d06373820eee09468c929fbb7a2c8688c0e0d155e187ad7bd0d72"}
Jul 20 20:39:04.925760 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.925764    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw" event={"ID":"bffab009-93ae-428b-bf97-790fd61322a1","Type":"ContainerStarted","Data":"91d406d4d0e08225eab036d672b4ad68b66d45c8189002b76d2e3ea611cc45ac"}
Jul 20 20:39:04.942281 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:04.942234    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw" podStartSLOduration=1.942220656 podStartE2EDuration="1.942220656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:39:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:39:04.941504144 +0000 UTC m=+2560.205574203" watchObservedRunningTime="2026-07-20 20:39:04.942220656 +0000 UTC m=+2560.206290702"
Jul 20 20:39:14.232353 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:14.232315    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:14.232760 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:14.232373    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:14.244806 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:14.244782    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:39:14.981698 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:39:14.981669    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:41:23.248458 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:23.248378    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"]
Jul 20 20:41:24.218071 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:24.218034    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"]
Jul 20 20:41:25.235320 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:25.235288    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 20:41:25.244697 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:25.244675    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ldg4_1a0e6c85-e495-4711-a23e-acdb2c031587/ovn-acl-logging/0.log"
Jul 20 20:41:38.267854 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.267798    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f" podUID="db71dbc0-0fd2-43df-9abd-8e63a2b28e80" containerName="main" containerID="cri-o://0ef61af3645a7df6f985804477e32a0d2b89a151ff66ab163683e48d70502053" gracePeriod=45
Jul 20 20:41:38.510880 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.510858    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:41:38.580883 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.580859    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-tmp-dir\") pod \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") "
Jul 20 20:41:38.581089 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.580917    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-model-cache\") pod \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") "
Jul 20 20:41:38.581089 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.580953    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-home\") pod \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") "
Jul 20 20:41:38.581089 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.580996    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-dshm\") pod \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") "
Jul 20 20:41:38.581089 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.581021    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxnln\" (UniqueName: \"kubernetes.io/projected/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-kube-api-access-pxnln\") pod \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") "
Jul 20 20:41:38.581089 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.581040    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-tls-certs\") pod \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\" (UID: \"db71dbc0-0fd2-43df-9abd-8e63a2b28e80\") "
Jul 20 20:41:38.581361 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.581219    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "db71dbc0-0fd2-43df-9abd-8e63a2b28e80" (UID: "db71dbc0-0fd2-43df-9abd-8e63a2b28e80"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:41:38.581361 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.581234    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-home" (OuterVolumeSpecName: "home") pod "db71dbc0-0fd2-43df-9abd-8e63a2b28e80" (UID: "db71dbc0-0fd2-43df-9abd-8e63a2b28e80"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:41:38.581361 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.581307    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-home\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:41:38.581361 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.581328    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-tmp-dir\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:41:38.581564 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.581435    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-model-cache" (OuterVolumeSpecName: "model-cache") pod "db71dbc0-0fd2-43df-9abd-8e63a2b28e80" (UID: "db71dbc0-0fd2-43df-9abd-8e63a2b28e80"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:41:38.583369 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.583344    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-dshm" (OuterVolumeSpecName: "dshm") pod "db71dbc0-0fd2-43df-9abd-8e63a2b28e80" (UID: "db71dbc0-0fd2-43df-9abd-8e63a2b28e80"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:41:38.583504 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.583418    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-kube-api-access-pxnln" (OuterVolumeSpecName: "kube-api-access-pxnln") pod "db71dbc0-0fd2-43df-9abd-8e63a2b28e80" (UID: "db71dbc0-0fd2-43df-9abd-8e63a2b28e80"). InnerVolumeSpecName "kube-api-access-pxnln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:41:38.583549 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.583501    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "db71dbc0-0fd2-43df-9abd-8e63a2b28e80" (UID: "db71dbc0-0fd2-43df-9abd-8e63a2b28e80"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 20:41:38.589308 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.589134    2580 generic.go:356] "Generic (PLEG): container finished" podID="db71dbc0-0fd2-43df-9abd-8e63a2b28e80" containerID="0ef61af3645a7df6f985804477e32a0d2b89a151ff66ab163683e48d70502053" exitCode=0
Jul 20 20:41:38.589308 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.589191    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"
Jul 20 20:41:38.589308 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.589214    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f" event={"ID":"db71dbc0-0fd2-43df-9abd-8e63a2b28e80","Type":"ContainerDied","Data":"0ef61af3645a7df6f985804477e32a0d2b89a151ff66ab163683e48d70502053"}
Jul 20 20:41:38.589308 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.589262    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f" event={"ID":"db71dbc0-0fd2-43df-9abd-8e63a2b28e80","Type":"ContainerDied","Data":"cb9d0dba35e5a35812bfc558db05c6100d2ce91a033170d89b81ee3578829ccd"}
Jul 20 20:41:38.589308 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.589284    2580 scope.go:117] "RemoveContainer" containerID="0ef61af3645a7df6f985804477e32a0d2b89a151ff66ab163683e48d70502053"
Jul 20 20:41:38.600505 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.600491    2580 scope.go:117] "RemoveContainer" containerID="0ef61af3645a7df6f985804477e32a0d2b89a151ff66ab163683e48d70502053"
Jul 20 20:41:38.600752 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:41:38.600735    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ef61af3645a7df6f985804477e32a0d2b89a151ff66ab163683e48d70502053\": container with ID starting with 0ef61af3645a7df6f985804477e32a0d2b89a151ff66ab163683e48d70502053 not found: ID does not exist" containerID="0ef61af3645a7df6f985804477e32a0d2b89a151ff66ab163683e48d70502053"
Jul 20 20:41:38.600803 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.600763    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ef61af3645a7df6f985804477e32a0d2b89a151ff66ab163683e48d70502053"} err="failed to get container status \"0ef61af3645a7df6f985804477e32a0d2b89a151ff66ab163683e48d70502053\": rpc error: code = NotFound desc = could not find container \"0ef61af3645a7df6f985804477e32a0d2b89a151ff66ab163683e48d70502053\": container with ID starting with 0ef61af3645a7df6f985804477e32a0d2b89a151ff66ab163683e48d70502053 not found: ID does not exist"
Jul 20 20:41:38.612145 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.612124    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"]
Jul 20 20:41:38.615823 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.615795    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b15c6d/auth-test-model-b15c6d-kserve-658cc49bdc-2w56f"]
Jul 20 20:41:38.682249 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.682221    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-model-cache\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:41:38.682249 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.682242    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-dshm\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:41:38.682249 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.682250    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxnln\" (UniqueName: \"kubernetes.io/projected/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-kube-api-access-pxnln\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:41:38.682430 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:38.682259    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db71dbc0-0fd2-43df-9abd-8e63a2b28e80-tls-certs\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:41:39.175343 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.175311    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db71dbc0-0fd2-43df-9abd-8e63a2b28e80" path="/var/lib/kubelet/pods/db71dbc0-0fd2-43df-9abd-8e63a2b28e80/volumes"
Jul 20 20:41:39.238147 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.238111    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw" podUID="bffab009-93ae-428b-bf97-790fd61322a1" containerName="main" containerID="cri-o://b50ba928470d06373820eee09468c929fbb7a2c8688c0e0d155e187ad7bd0d72" gracePeriod=45
Jul 20 20:41:39.476272 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.476249    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:41:39.589168 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.589134    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-home\") pod \"bffab009-93ae-428b-bf97-790fd61322a1\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") "
Jul 20 20:41:39.589168 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.589173    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-846jd\" (UniqueName: \"kubernetes.io/projected/bffab009-93ae-428b-bf97-790fd61322a1-kube-api-access-846jd\") pod \"bffab009-93ae-428b-bf97-790fd61322a1\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") "
Jul 20 20:41:39.589398 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.589232    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-model-cache\") pod \"bffab009-93ae-428b-bf97-790fd61322a1\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") "
Jul 20 20:41:39.589398 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.589262    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bffab009-93ae-428b-bf97-790fd61322a1-tls-certs\") pod \"bffab009-93ae-428b-bf97-790fd61322a1\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") "
Jul 20 20:41:39.589398 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.589286    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-dshm\") pod \"bffab009-93ae-428b-bf97-790fd61322a1\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") "
Jul 20 20:41:39.589398 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.589323    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-tmp-dir\") pod \"bffab009-93ae-428b-bf97-790fd61322a1\" (UID: \"bffab009-93ae-428b-bf97-790fd61322a1\") "
Jul 20 20:41:39.589624 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.589475    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-home" (OuterVolumeSpecName: "home") pod "bffab009-93ae-428b-bf97-790fd61322a1" (UID: "bffab009-93ae-428b-bf97-790fd61322a1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:41:39.589624 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.589577    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-model-cache" (OuterVolumeSpecName: "model-cache") pod "bffab009-93ae-428b-bf97-790fd61322a1" (UID: "bffab009-93ae-428b-bf97-790fd61322a1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:41:39.589624 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.589597    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-home\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:41:39.589773 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.589666    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bffab009-93ae-428b-bf97-790fd61322a1" (UID: "bffab009-93ae-428b-bf97-790fd61322a1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:41:39.591595 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.591566    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bffab009-93ae-428b-bf97-790fd61322a1-kube-api-access-846jd" (OuterVolumeSpecName: "kube-api-access-846jd") pod "bffab009-93ae-428b-bf97-790fd61322a1" (UID: "bffab009-93ae-428b-bf97-790fd61322a1"). InnerVolumeSpecName "kube-api-access-846jd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:41:39.591864 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.591839    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bffab009-93ae-428b-bf97-790fd61322a1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bffab009-93ae-428b-bf97-790fd61322a1" (UID: "bffab009-93ae-428b-bf97-790fd61322a1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 20:41:39.591954 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.591863    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-dshm" (OuterVolumeSpecName: "dshm") pod "bffab009-93ae-428b-bf97-790fd61322a1" (UID: "bffab009-93ae-428b-bf97-790fd61322a1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:41:39.601496 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.601471    2580 generic.go:356] "Generic (PLEG): container finished" podID="bffab009-93ae-428b-bf97-790fd61322a1" containerID="b50ba928470d06373820eee09468c929fbb7a2c8688c0e0d155e187ad7bd0d72" exitCode=0
Jul 20 20:41:39.601582 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.601522    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw" event={"ID":"bffab009-93ae-428b-bf97-790fd61322a1","Type":"ContainerDied","Data":"b50ba928470d06373820eee09468c929fbb7a2c8688c0e0d155e187ad7bd0d72"}
Jul 20 20:41:39.601582 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.601530    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"
Jul 20 20:41:39.601582 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.601561    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw" event={"ID":"bffab009-93ae-428b-bf97-790fd61322a1","Type":"ContainerDied","Data":"91d406d4d0e08225eab036d672b4ad68b66d45c8189002b76d2e3ea611cc45ac"}
Jul 20 20:41:39.601688 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.601584    2580 scope.go:117] "RemoveContainer" containerID="b50ba928470d06373820eee09468c929fbb7a2c8688c0e0d155e187ad7bd0d72"
Jul 20 20:41:39.610414 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.610393    2580 scope.go:117] "RemoveContainer" containerID="b50ba928470d06373820eee09468c929fbb7a2c8688c0e0d155e187ad7bd0d72"
Jul 20 20:41:39.610670 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:41:39.610650    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b50ba928470d06373820eee09468c929fbb7a2c8688c0e0d155e187ad7bd0d72\": container with ID starting with b50ba928470d06373820eee09468c929fbb7a2c8688c0e0d155e187ad7bd0d72 not found: ID does not exist" containerID="b50ba928470d06373820eee09468c929fbb7a2c8688c0e0d155e187ad7bd0d72"
Jul 20 20:41:39.610722 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.610677    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b50ba928470d06373820eee09468c929fbb7a2c8688c0e0d155e187ad7bd0d72"} err="failed to get container status \"b50ba928470d06373820eee09468c929fbb7a2c8688c0e0d155e187ad7bd0d72\": rpc error: code = NotFound desc = could not find container \"b50ba928470d06373820eee09468c929fbb7a2c8688c0e0d155e187ad7bd0d72\": container with ID starting with b50ba928470d06373820eee09468c929fbb7a2c8688c0e0d155e187ad7bd0d72 not found: ID does not exist"
Jul 20 20:41:39.634628 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.634605    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"]
Jul 20 20:41:39.640222 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.640203    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-40065f/auth-test-model-40065f-kserve-7f8fd4bb9f-p86nw"]
Jul 20 20:41:39.690705 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.690682    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-tmp-dir\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:41:39.690705 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.690703    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-846jd\" (UniqueName: \"kubernetes.io/projected/bffab009-93ae-428b-bf97-790fd61322a1-kube-api-access-846jd\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:41:39.690838 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.690713    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-model-cache\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:41:39.690838 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.690722    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bffab009-93ae-428b-bf97-790fd61322a1-tls-certs\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:41:39.690838 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:39.690730    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bffab009-93ae-428b-bf97-790fd61322a1-dshm\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:41:41.175569 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.175538    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bffab009-93ae-428b-bf97-790fd61322a1" path="/var/lib/kubelet/pods/bffab009-93ae-428b-bf97-790fd61322a1/volumes"
Jul 20 20:41:41.531915 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.531835    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"]
Jul 20 20:41:41.532142 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.532129    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db71dbc0-0fd2-43df-9abd-8e63a2b28e80" containerName="main"
Jul 20 20:41:41.532183 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.532143    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="db71dbc0-0fd2-43df-9abd-8e63a2b28e80" containerName="main"
Jul 20 20:41:41.532183 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.532164    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bffab009-93ae-428b-bf97-790fd61322a1" containerName="main"
Jul 20 20:41:41.532183 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.532170    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bffab009-93ae-428b-bf97-790fd61322a1" containerName="main"
Jul 20 20:41:41.532276 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.532212    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="db71dbc0-0fd2-43df-9abd-8e63a2b28e80" containerName="main"
Jul 20 20:41:41.532276 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.532222    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bffab009-93ae-428b-bf97-790fd61322a1" containerName="main"
Jul 20 20:41:41.536446 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.536427    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:41.539639 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.539616    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b15c6d\"/\"openshift-service-ca.crt\""
Jul 20 20:41:41.539838 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.539823    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b15c6d\"/\"kube-root-ca.crt\""
Jul 20 20:41:41.541164 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.541145    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b15c6d\"/\"sub-test-model-b15c6d-kserve-self-signed-certs\""
Jul 20 20:41:41.541253 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.541196    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b15c6d\"/\"default-dockercfg-6v24x\""
Jul 20 20:41:41.545316 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.545295    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"]
Jul 20 20:41:41.607181 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.607154    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-dshm\") pod \"sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") " pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:41.607181 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.607182    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-model-cache\") pod \"sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") " pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:41.607385 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.607204    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-tmp-dir\") pod \"sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") " pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:41.607433 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.607413    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bpc2\" (UniqueName: \"kubernetes.io/projected/54612f9c-ea68-44e3-9521-979aa279838a-kube-api-access-2bpc2\") pod \"sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") " pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:41.607479 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.607454    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-home\") pod \"sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") " pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:41.607522 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.607494    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54612f9c-ea68-44e3-9521-979aa279838a-tls-certs\") pod \"sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") " pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:41.708578 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.708542    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-dshm\") pod \"sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") " pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:41.708578 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.708576    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-model-cache\") pod \"sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") " pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:41.708817 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.708595    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-tmp-dir\") pod \"sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") " pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:41.708817 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.708644    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bpc2\" (UniqueName: \"kubernetes.io/projected/54612f9c-ea68-44e3-9521-979aa279838a-kube-api-access-2bpc2\") pod \"sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") " pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:41.708817 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.708663    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-home\") pod \"sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") " pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:41.708817 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.708683    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54612f9c-ea68-44e3-9521-979aa279838a-tls-certs\") pod \"sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") " pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:41.709096 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.709065    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-model-cache\") pod \"sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") " pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:41.709146 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.709105    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-tmp-dir\") pod \"sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") " pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:41.709183 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.709145    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-home\") pod \"sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") " pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:41.710830 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.710809    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-dshm\") pod \"sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") " pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:41.711100 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.711084    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54612f9c-ea68-44e3-9521-979aa279838a-tls-certs\") pod \"sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") " pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:41.717605 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.717581    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bpc2\" (UniqueName: \"kubernetes.io/projected/54612f9c-ea68-44e3-9521-979aa279838a-kube-api-access-2bpc2\") pod \"sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") " pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:41.845651 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.845617    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:41.975539 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:41.975514    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"]
Jul 20 20:41:41.978123 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:41:41.978099    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod54612f9c_ea68_44e3_9521_979aa279838a.slice/crio-52d30bfb3c289b3480473e6777932e5018a145f7f171377684f0594d0704174f WatchSource:0}: Error finding container 52d30bfb3c289b3480473e6777932e5018a145f7f171377684f0594d0704174f: Status 404 returned error can't find the container with id 52d30bfb3c289b3480473e6777932e5018a145f7f171377684f0594d0704174f
Jul 20 20:41:42.616784 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:42.616746    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg" event={"ID":"54612f9c-ea68-44e3-9521-979aa279838a","Type":"ContainerStarted","Data":"23fe73597c317dfbe1343a109f04b005ac3ed33e8bee7bb846d73dc542f2369c"}
Jul 20 20:41:42.616784 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:42.616787    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg" event={"ID":"54612f9c-ea68-44e3-9521-979aa279838a","Type":"ContainerStarted","Data":"52d30bfb3c289b3480473e6777932e5018a145f7f171377684f0594d0704174f"}
Jul 20 20:41:42.635262 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:42.635215    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg" podStartSLOduration=1.635200684 podStartE2EDuration="1.635200684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:41:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:41:42.634130787 +0000 UTC m=+2717.898200844" watchObservedRunningTime="2026-07-20 20:41:42.635200684 +0000 UTC m=+2717.899270729"
Jul 20 20:41:51.845861 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:51.845820    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:51.846276 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:51.845877    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:51.858445 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:51.858420    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:41:52.668711 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:41:52.668682    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:42:08.436298 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.436253    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"]
Jul 20 20:42:08.440025 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.440001    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:08.446202 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.446175    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-40065f\"/\"kube-root-ca.crt\""
Jul 20 20:42:08.452198 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.452175    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-40065f\"/\"sub-test-model-40065f-kserve-self-signed-certs\""
Jul 20 20:42:08.452312 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.452203    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-40065f\"/\"default-dockercfg-84tz8\""
Jul 20 20:42:08.452453 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.452439    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-40065f\"/\"openshift-service-ca.crt\""
Jul 20 20:42:08.453920 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.453900    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"]
Jul 20 20:42:08.538565 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.538529    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df55f393-2925-45d4-a026-b1b413d7dfa2-tls-certs\") pod \"sub-test-model-40065f-kserve-576f7dfcc-v9hsj\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") " pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:08.538771 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.538572    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-model-cache\") pod \"sub-test-model-40065f-kserve-576f7dfcc-v9hsj\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") " pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:08.538771 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.538657    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-home\") pod \"sub-test-model-40065f-kserve-576f7dfcc-v9hsj\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") " pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:08.538771 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.538721    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-dshm\") pod \"sub-test-model-40065f-kserve-576f7dfcc-v9hsj\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") " pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:08.538771 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.538752    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-tmp-dir\") pod \"sub-test-model-40065f-kserve-576f7dfcc-v9hsj\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") " pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:08.538924 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.538831    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csdc2\" (UniqueName: \"kubernetes.io/projected/df55f393-2925-45d4-a026-b1b413d7dfa2-kube-api-access-csdc2\") pod \"sub-test-model-40065f-kserve-576f7dfcc-v9hsj\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") " pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:08.639671 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.639631    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-dshm\") pod \"sub-test-model-40065f-kserve-576f7dfcc-v9hsj\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") " pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:08.639867 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.639689    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-tmp-dir\") pod \"sub-test-model-40065f-kserve-576f7dfcc-v9hsj\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") " pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:08.639867 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.639777    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csdc2\" (UniqueName: \"kubernetes.io/projected/df55f393-2925-45d4-a026-b1b413d7dfa2-kube-api-access-csdc2\") pod \"sub-test-model-40065f-kserve-576f7dfcc-v9hsj\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") " pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:08.639867 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.639814    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df55f393-2925-45d4-a026-b1b413d7dfa2-tls-certs\") pod \"sub-test-model-40065f-kserve-576f7dfcc-v9hsj\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") " pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:08.639867 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.639849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-model-cache\") pod \"sub-test-model-40065f-kserve-576f7dfcc-v9hsj\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") " pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:08.640112 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.639890    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-home\") pod \"sub-test-model-40065f-kserve-576f7dfcc-v9hsj\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") " pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:08.640169 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.640145    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-tmp-dir\") pod \"sub-test-model-40065f-kserve-576f7dfcc-v9hsj\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") " pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:08.640216 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.640170    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-model-cache\") pod \"sub-test-model-40065f-kserve-576f7dfcc-v9hsj\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") " pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:08.640377 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.640349    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-home\") pod \"sub-test-model-40065f-kserve-576f7dfcc-v9hsj\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") " pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:08.642072 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.642051    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-dshm\") pod \"sub-test-model-40065f-kserve-576f7dfcc-v9hsj\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") " pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:08.642242 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.642227    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df55f393-2925-45d4-a026-b1b413d7dfa2-tls-certs\") pod \"sub-test-model-40065f-kserve-576f7dfcc-v9hsj\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") " pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:08.649327 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.649306    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csdc2\" (UniqueName: \"kubernetes.io/projected/df55f393-2925-45d4-a026-b1b413d7dfa2-kube-api-access-csdc2\") pod \"sub-test-model-40065f-kserve-576f7dfcc-v9hsj\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") " pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:08.750252 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.750164    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:08.877480 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:08.877457    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"]
Jul 20 20:42:08.879837 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:42:08.879811    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf55f393_2925_45d4_a026_b1b413d7dfa2.slice/crio-62e30381103458e7edf4e9a8cabe360a25aee660919b87f985adcf19801ad1cb WatchSource:0}: Error finding container 62e30381103458e7edf4e9a8cabe360a25aee660919b87f985adcf19801ad1cb: Status 404 returned error can't find the container with id 62e30381103458e7edf4e9a8cabe360a25aee660919b87f985adcf19801ad1cb
Jul 20 20:42:09.737239 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:09.737203    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj" event={"ID":"df55f393-2925-45d4-a026-b1b413d7dfa2","Type":"ContainerStarted","Data":"143cec2203f149c77795ccd70e7a10b61f6daa7a2e390b06f5e447d110415f66"}
Jul 20 20:42:09.737609 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:09.737250    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj" event={"ID":"df55f393-2925-45d4-a026-b1b413d7dfa2","Type":"ContainerStarted","Data":"62e30381103458e7edf4e9a8cabe360a25aee660919b87f985adcf19801ad1cb"}
Jul 20 20:42:09.766589 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:09.766535    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj" podStartSLOduration=1.766519913 podStartE2EDuration="1.766519913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:42:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:42:09.765568382 +0000 UTC m=+2745.029638430" watchObservedRunningTime="2026-07-20 20:42:09.766519913 +0000 UTC m=+2745.030589958"
Jul 20 20:42:18.750522 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:18.750482    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:18.750522 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:18.750528    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:18.762906 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:18.762878    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:18.787837 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:18.787808    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:42:44.415745 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:44.415705    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"]
Jul 20 20:42:46.139606 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:46.139577    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"]
Jul 20 20:42:59.436257 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.436157    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg" podUID="54612f9c-ea68-44e3-9521-979aa279838a" containerName="main" containerID="cri-o://23fe73597c317dfbe1343a109f04b005ac3ed33e8bee7bb846d73dc542f2369c" gracePeriod=45
Jul 20 20:42:59.681466 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.681441    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:42:59.784389 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.784300    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54612f9c-ea68-44e3-9521-979aa279838a-tls-certs\") pod \"54612f9c-ea68-44e3-9521-979aa279838a\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") "
Jul 20 20:42:59.784567 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.784401    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-tmp-dir\") pod \"54612f9c-ea68-44e3-9521-979aa279838a\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") "
Jul 20 20:42:59.784567 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.784421    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-model-cache\") pod \"54612f9c-ea68-44e3-9521-979aa279838a\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") "
Jul 20 20:42:59.784567 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.784437    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-home\") pod \"54612f9c-ea68-44e3-9521-979aa279838a\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") "
Jul 20 20:42:59.784567 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.784463    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-dshm\") pod \"54612f9c-ea68-44e3-9521-979aa279838a\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") "
Jul 20 20:42:59.784567 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.784489    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bpc2\" (UniqueName: \"kubernetes.io/projected/54612f9c-ea68-44e3-9521-979aa279838a-kube-api-access-2bpc2\") pod \"54612f9c-ea68-44e3-9521-979aa279838a\" (UID: \"54612f9c-ea68-44e3-9521-979aa279838a\") "
Jul 20 20:42:59.784824 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.784721    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-home" (OuterVolumeSpecName: "home") pod "54612f9c-ea68-44e3-9521-979aa279838a" (UID: "54612f9c-ea68-44e3-9521-979aa279838a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:42:59.784824 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.784735    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "54612f9c-ea68-44e3-9521-979aa279838a" (UID: "54612f9c-ea68-44e3-9521-979aa279838a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:42:59.784824 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.784722    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-model-cache" (OuterVolumeSpecName: "model-cache") pod "54612f9c-ea68-44e3-9521-979aa279838a" (UID: "54612f9c-ea68-44e3-9521-979aa279838a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:42:59.786572 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.786550    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54612f9c-ea68-44e3-9521-979aa279838a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "54612f9c-ea68-44e3-9521-979aa279838a" (UID: "54612f9c-ea68-44e3-9521-979aa279838a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 20:42:59.786662 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.786579    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-dshm" (OuterVolumeSpecName: "dshm") pod "54612f9c-ea68-44e3-9521-979aa279838a" (UID: "54612f9c-ea68-44e3-9521-979aa279838a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:42:59.786662 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.786593    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54612f9c-ea68-44e3-9521-979aa279838a-kube-api-access-2bpc2" (OuterVolumeSpecName: "kube-api-access-2bpc2") pod "54612f9c-ea68-44e3-9521-979aa279838a" (UID: "54612f9c-ea68-44e3-9521-979aa279838a"). InnerVolumeSpecName "kube-api-access-2bpc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:42:59.886035 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.886001    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-tmp-dir\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:42:59.886035 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.886029    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-model-cache\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:42:59.886035 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.886038    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-home\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:42:59.886035 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.886047    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54612f9c-ea68-44e3-9521-979aa279838a-dshm\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:42:59.886305 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.886055    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bpc2\" (UniqueName: \"kubernetes.io/projected/54612f9c-ea68-44e3-9521-979aa279838a-kube-api-access-2bpc2\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:42:59.886305 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.886063    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54612f9c-ea68-44e3-9521-979aa279838a-tls-certs\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:42:59.950162 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.950126    2580 generic.go:356] "Generic (PLEG): container finished" podID="54612f9c-ea68-44e3-9521-979aa279838a" containerID="23fe73597c317dfbe1343a109f04b005ac3ed33e8bee7bb846d73dc542f2369c" exitCode=0
Jul 20 20:42:59.950310 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.950177    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"
Jul 20 20:42:59.950310 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.950215    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg" event={"ID":"54612f9c-ea68-44e3-9521-979aa279838a","Type":"ContainerDied","Data":"23fe73597c317dfbe1343a109f04b005ac3ed33e8bee7bb846d73dc542f2369c"}
Jul 20 20:42:59.950310 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.950263    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg" event={"ID":"54612f9c-ea68-44e3-9521-979aa279838a","Type":"ContainerDied","Data":"52d30bfb3c289b3480473e6777932e5018a145f7f171377684f0594d0704174f"}
Jul 20 20:42:59.950310 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.950280    2580 scope.go:117] "RemoveContainer" containerID="23fe73597c317dfbe1343a109f04b005ac3ed33e8bee7bb846d73dc542f2369c"
Jul 20 20:42:59.958893 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.958876    2580 scope.go:117] "RemoveContainer" containerID="23fe73597c317dfbe1343a109f04b005ac3ed33e8bee7bb846d73dc542f2369c"
Jul 20 20:42:59.959175 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:42:59.959154    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23fe73597c317dfbe1343a109f04b005ac3ed33e8bee7bb846d73dc542f2369c\": container with ID starting with 23fe73597c317dfbe1343a109f04b005ac3ed33e8bee7bb846d73dc542f2369c not found: ID does not exist" containerID="23fe73597c317dfbe1343a109f04b005ac3ed33e8bee7bb846d73dc542f2369c"
Jul 20 20:42:59.959219 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.959183    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23fe73597c317dfbe1343a109f04b005ac3ed33e8bee7bb846d73dc542f2369c"} err="failed to get container status \"23fe73597c317dfbe1343a109f04b005ac3ed33e8bee7bb846d73dc542f2369c\": rpc error: code = NotFound desc = could not find container \"23fe73597c317dfbe1343a109f04b005ac3ed33e8bee7bb846d73dc542f2369c\": container with ID starting with 23fe73597c317dfbe1343a109f04b005ac3ed33e8bee7bb846d73dc542f2369c not found: ID does not exist"
Jul 20 20:42:59.970441 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.970416    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"]
Jul 20 20:42:59.974786 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:42:59.974765    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b15c6d/sub-test-model-b15c6d-kserve-64c7fd8d6f-w5zdg"]
Jul 20 20:43:01.159298 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.159257    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj" podUID="df55f393-2925-45d4-a026-b1b413d7dfa2" containerName="main" containerID="cri-o://143cec2203f149c77795ccd70e7a10b61f6daa7a2e390b06f5e447d110415f66" gracePeriod=45
Jul 20 20:43:01.175249 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.175226    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54612f9c-ea68-44e3-9521-979aa279838a" path="/var/lib/kubelet/pods/54612f9c-ea68-44e3-9521-979aa279838a/volumes"
Jul 20 20:43:01.405234 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.405211    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:43:01.498997 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.498896    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-home\") pod \"df55f393-2925-45d4-a026-b1b413d7dfa2\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") "
Jul 20 20:43:01.498997 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.498936    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-tmp-dir\") pod \"df55f393-2925-45d4-a026-b1b413d7dfa2\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") "
Jul 20 20:43:01.499216 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.499037    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-dshm\") pod \"df55f393-2925-45d4-a026-b1b413d7dfa2\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") "
Jul 20 20:43:01.499216 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.499067    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df55f393-2925-45d4-a026-b1b413d7dfa2-tls-certs\") pod \"df55f393-2925-45d4-a026-b1b413d7dfa2\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") "
Jul 20 20:43:01.499216 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.499095    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-csdc2\" (UniqueName: \"kubernetes.io/projected/df55f393-2925-45d4-a026-b1b413d7dfa2-kube-api-access-csdc2\") pod \"df55f393-2925-45d4-a026-b1b413d7dfa2\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") "
Jul 20 20:43:01.499216 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.499194    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-model-cache\") pod \"df55f393-2925-45d4-a026-b1b413d7dfa2\" (UID: \"df55f393-2925-45d4-a026-b1b413d7dfa2\") "
Jul 20 20:43:01.499216 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.499199    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-home" (OuterVolumeSpecName: "home") pod "df55f393-2925-45d4-a026-b1b413d7dfa2" (UID: "df55f393-2925-45d4-a026-b1b413d7dfa2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:43:01.499438 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.499299    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "df55f393-2925-45d4-a026-b1b413d7dfa2" (UID: "df55f393-2925-45d4-a026-b1b413d7dfa2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:43:01.499438 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.499315    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-model-cache" (OuterVolumeSpecName: "model-cache") pod "df55f393-2925-45d4-a026-b1b413d7dfa2" (UID: "df55f393-2925-45d4-a026-b1b413d7dfa2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:43:01.499543 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.499479    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-model-cache\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:43:01.499543 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.499491    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-home\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:43:01.499543 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.499499    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-tmp-dir\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:43:01.501275 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.501249    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-dshm" (OuterVolumeSpecName: "dshm") pod "df55f393-2925-45d4-a026-b1b413d7dfa2" (UID: "df55f393-2925-45d4-a026-b1b413d7dfa2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:43:01.501380 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.501271    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df55f393-2925-45d4-a026-b1b413d7dfa2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "df55f393-2925-45d4-a026-b1b413d7dfa2" (UID: "df55f393-2925-45d4-a026-b1b413d7dfa2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 20:43:01.501380 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.501303    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df55f393-2925-45d4-a026-b1b413d7dfa2-kube-api-access-csdc2" (OuterVolumeSpecName: "kube-api-access-csdc2") pod "df55f393-2925-45d4-a026-b1b413d7dfa2" (UID: "df55f393-2925-45d4-a026-b1b413d7dfa2"). InnerVolumeSpecName "kube-api-access-csdc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:43:01.600828 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.600796    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df55f393-2925-45d4-a026-b1b413d7dfa2-dshm\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:43:01.600828 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.600820    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df55f393-2925-45d4-a026-b1b413d7dfa2-tls-certs\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:43:01.600828 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.600831    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-csdc2\" (UniqueName: \"kubernetes.io/projected/df55f393-2925-45d4-a026-b1b413d7dfa2-kube-api-access-csdc2\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:43:01.961020 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.960986    2580 generic.go:356] "Generic (PLEG): container finished" podID="df55f393-2925-45d4-a026-b1b413d7dfa2" containerID="143cec2203f149c77795ccd70e7a10b61f6daa7a2e390b06f5e447d110415f66" exitCode=0
Jul 20 20:43:01.961020 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.961024    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"
Jul 20 20:43:01.961257 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.961050    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj" event={"ID":"df55f393-2925-45d4-a026-b1b413d7dfa2","Type":"ContainerDied","Data":"143cec2203f149c77795ccd70e7a10b61f6daa7a2e390b06f5e447d110415f66"}
Jul 20 20:43:01.961257 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.961081    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj" event={"ID":"df55f393-2925-45d4-a026-b1b413d7dfa2","Type":"ContainerDied","Data":"62e30381103458e7edf4e9a8cabe360a25aee660919b87f985adcf19801ad1cb"}
Jul 20 20:43:01.961257 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.961099    2580 scope.go:117] "RemoveContainer" containerID="143cec2203f149c77795ccd70e7a10b61f6daa7a2e390b06f5e447d110415f66"
Jul 20 20:43:01.969941 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.969923    2580 scope.go:117] "RemoveContainer" containerID="143cec2203f149c77795ccd70e7a10b61f6daa7a2e390b06f5e447d110415f66"
Jul 20 20:43:01.970267 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:43:01.970244    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"143cec2203f149c77795ccd70e7a10b61f6daa7a2e390b06f5e447d110415f66\": container with ID starting with 143cec2203f149c77795ccd70e7a10b61f6daa7a2e390b06f5e447d110415f66 not found: ID does not exist" containerID="143cec2203f149c77795ccd70e7a10b61f6daa7a2e390b06f5e447d110415f66"
Jul 20 20:43:01.970346 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.970279    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"143cec2203f149c77795ccd70e7a10b61f6daa7a2e390b06f5e447d110415f66"} err="failed to get container status \"143cec2203f149c77795ccd70e7a10b61f6daa7a2e390b06f5e447d110415f66\": rpc error: code = NotFound desc = could not find container \"143cec2203f149c77795ccd70e7a10b61f6daa7a2e390b06f5e447d110415f66\": container with ID starting with 143cec2203f149c77795ccd70e7a10b61f6daa7a2e390b06f5e447d110415f66 not found: ID does not exist"
Jul 20 20:43:01.981182 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.981151    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"]
Jul 20 20:43:01.985565 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:01.985540    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-40065f/sub-test-model-40065f-kserve-576f7dfcc-v9hsj"]
Jul 20 20:43:03.175698 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:03.175661    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df55f393-2925-45d4-a026-b1b413d7dfa2" path="/var/lib/kubelet/pods/df55f393-2925-45d4-a026-b1b413d7dfa2/volumes"
Jul 20 20:43:07.033512 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.033475    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"]
Jul 20 20:43:07.034001 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.033910    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54612f9c-ea68-44e3-9521-979aa279838a" containerName="main"
Jul 20 20:43:07.034001 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.033996    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="54612f9c-ea68-44e3-9521-979aa279838a" containerName="main"
Jul 20 20:43:07.034134 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.034028    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df55f393-2925-45d4-a026-b1b413d7dfa2" containerName="main"
Jul 20 20:43:07.034134 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.034037    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="df55f393-2925-45d4-a026-b1b413d7dfa2" containerName="main"
Jul 20 20:43:07.034231 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.034147    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="54612f9c-ea68-44e3-9521-979aa279838a" containerName="main"
Jul 20 20:43:07.034231 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.034160    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="df55f393-2925-45d4-a026-b1b413d7dfa2" containerName="main"
Jul 20 20:43:07.036291 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.036272    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:07.040224 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.040202    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b15c6d\"/\"rate-test-model-b15c6d-kserve-self-signed-certs\""
Jul 20 20:43:07.040324 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.040233    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b15c6d\"/\"kube-root-ca.crt\""
Jul 20 20:43:07.040324 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.040252    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b15c6d\"/\"openshift-service-ca.crt\""
Jul 20 20:43:07.040324 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.040262    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b15c6d\"/\"default-dockercfg-6v24x\""
Jul 20 20:43:07.045297 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.045276    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"]
Jul 20 20:43:07.147333 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.147292    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-model-cache\") pod \"rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") " pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:07.147503 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.147350    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-home\") pod \"rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") " pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:07.147503 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.147437    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-tmp-dir\") pod \"rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") " pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:07.147503 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.147469    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-dshm\") pod \"rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") " pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:07.147503 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.147488    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e7c6c28-059b-4b48-8c72-938466e1acb9-tls-certs\") pod \"rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") " pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:07.147671 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.147511    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-287tk\" (UniqueName: \"kubernetes.io/projected/5e7c6c28-059b-4b48-8c72-938466e1acb9-kube-api-access-287tk\") pod \"rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") " pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:07.248111 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.248076    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-home\") pod \"rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") " pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:07.248317 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.248152    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-tmp-dir\") pod \"rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") " pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:07.248317 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.248178    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-dshm\") pod \"rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") " pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:07.248317 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.248203    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e7c6c28-059b-4b48-8c72-938466e1acb9-tls-certs\") pod \"rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") " pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:07.248317 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.248234    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-287tk\" (UniqueName: \"kubernetes.io/projected/5e7c6c28-059b-4b48-8c72-938466e1acb9-kube-api-access-287tk\") pod \"rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") " pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:07.248562 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.248432    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-model-cache\") pod \"rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") " pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:07.248562 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.248523    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-home\") pod \"rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") " pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:07.248674 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.248591    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-tmp-dir\") pod \"rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") " pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:07.248757 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.248738    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-model-cache\") pod \"rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") " pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:07.250346 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.250327    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-dshm\") pod \"rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") " pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:07.250702 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.250687    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e7c6c28-059b-4b48-8c72-938466e1acb9-tls-certs\") pod \"rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") " pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:07.256602 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.256571    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-287tk\" (UniqueName: \"kubernetes.io/projected/5e7c6c28-059b-4b48-8c72-938466e1acb9-kube-api-access-287tk\") pod \"rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") " pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:07.346790 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.346756    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:07.472707 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.472681    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"]
Jul 20 20:43:07.475510 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:43:07.475469    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e7c6c28_059b_4b48_8c72_938466e1acb9.slice/crio-be246ee0934dfb736db51a4b097d290c75126dcedb73ba86c72f8baca2f762c0 WatchSource:0}: Error finding container be246ee0934dfb736db51a4b097d290c75126dcedb73ba86c72f8baca2f762c0: Status 404 returned error can't find the container with id be246ee0934dfb736db51a4b097d290c75126dcedb73ba86c72f8baca2f762c0
Jul 20 20:43:07.988303 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.988263    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm" event={"ID":"5e7c6c28-059b-4b48-8c72-938466e1acb9","Type":"ContainerStarted","Data":"4ac355d4e3fef973cbc865faadd04470aca26940ddcb8eeba72b87fa39032052"}
Jul 20 20:43:07.988516 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:07.988317    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm" event={"ID":"5e7c6c28-059b-4b48-8c72-938466e1acb9","Type":"ContainerStarted","Data":"be246ee0934dfb736db51a4b097d290c75126dcedb73ba86c72f8baca2f762c0"}
Jul 20 20:43:08.012252 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:08.012200    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm" podStartSLOduration=1.012186046 podStartE2EDuration="1.012186046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:43:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:43:08.010539297 +0000 UTC m=+2803.274609343" watchObservedRunningTime="2026-07-20 20:43:08.012186046 +0000 UTC m=+2803.276256091"
Jul 20 20:43:17.346869 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:17.346828    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:17.349283 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:17.346886    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:17.359359 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:17.359337    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:18.042321 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:18.042283    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:43:25.452023 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:25.451990    2580 scope.go:117] "RemoveContainer" containerID="5f62176ea66bf325cc5462733da1b710c644e216c7ec62fea4ea9d2c189ee179"
Jul 20 20:43:25.459778 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:25.459761    2580 scope.go:117] "RemoveContainer" containerID="941a677245e96f1a68c742bd19d616696ef8c113731ccfbeaa8e410451f8f5d6"
Jul 20 20:43:37.832739 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:37.832707    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"]
Jul 20 20:43:37.835569 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:37.835550    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:37.838535 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:37.838503    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-40065f\"/\"openshift-service-ca.crt\""
Jul 20 20:43:37.838535 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:37.838516    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-40065f\"/\"kube-root-ca.crt\""
Jul 20 20:43:37.839774 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:37.839756    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-40065f\"/\"rate-test-model-40065f-kserve-self-signed-certs\""
Jul 20 20:43:37.839894 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:37.839812    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-40065f\"/\"default-dockercfg-84tz8\""
Jul 20 20:43:37.846875 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:37.846856    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"]
Jul 20 20:43:37.904315 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:37.904281    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-model-cache\") pod \"rate-test-model-40065f-kserve-76949f97cf-wvxq5\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") " pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:37.904482 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:37.904322    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9152e2fb-1e34-4256-b046-b28df9ab3b7c-tls-certs\") pod \"rate-test-model-40065f-kserve-76949f97cf-wvxq5\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") " pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:37.904482 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:37.904408    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-home\") pod \"rate-test-model-40065f-kserve-76949f97cf-wvxq5\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") " pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:37.904482 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:37.904453    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-tmp-dir\") pod \"rate-test-model-40065f-kserve-76949f97cf-wvxq5\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") " pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:37.904482 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:37.904478    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq98p\" (UniqueName: \"kubernetes.io/projected/9152e2fb-1e34-4256-b046-b28df9ab3b7c-kube-api-access-fq98p\") pod \"rate-test-model-40065f-kserve-76949f97cf-wvxq5\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") " pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:37.904630 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:37.904506    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-dshm\") pod \"rate-test-model-40065f-kserve-76949f97cf-wvxq5\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") " pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:38.005509 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:38.005469    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-tmp-dir\") pod \"rate-test-model-40065f-kserve-76949f97cf-wvxq5\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") " pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:38.005509 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:38.005514    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq98p\" (UniqueName: \"kubernetes.io/projected/9152e2fb-1e34-4256-b046-b28df9ab3b7c-kube-api-access-fq98p\") pod \"rate-test-model-40065f-kserve-76949f97cf-wvxq5\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") " pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:38.005760 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:38.005536    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-dshm\") pod \"rate-test-model-40065f-kserve-76949f97cf-wvxq5\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") " pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:38.005760 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:38.005560    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-model-cache\") pod \"rate-test-model-40065f-kserve-76949f97cf-wvxq5\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") " pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:38.005760 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:38.005583    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9152e2fb-1e34-4256-b046-b28df9ab3b7c-tls-certs\") pod \"rate-test-model-40065f-kserve-76949f97cf-wvxq5\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") " pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:38.005760 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:38.005643    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-home\") pod \"rate-test-model-40065f-kserve-76949f97cf-wvxq5\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") " pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:38.006248 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:38.005948    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-tmp-dir\") pod \"rate-test-model-40065f-kserve-76949f97cf-wvxq5\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") " pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:38.006248 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:38.006036    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-home\") pod \"rate-test-model-40065f-kserve-76949f97cf-wvxq5\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") " pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:38.006248 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:38.006055    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-model-cache\") pod \"rate-test-model-40065f-kserve-76949f97cf-wvxq5\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") " pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:38.007775 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:38.007759    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-dshm\") pod \"rate-test-model-40065f-kserve-76949f97cf-wvxq5\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") " pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:38.008051 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:38.008035    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9152e2fb-1e34-4256-b046-b28df9ab3b7c-tls-certs\") pod \"rate-test-model-40065f-kserve-76949f97cf-wvxq5\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") " pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:38.014671 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:38.014649    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq98p\" (UniqueName: \"kubernetes.io/projected/9152e2fb-1e34-4256-b046-b28df9ab3b7c-kube-api-access-fq98p\") pod \"rate-test-model-40065f-kserve-76949f97cf-wvxq5\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") " pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:38.145897 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:38.145807    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:38.284758 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:38.284731    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"]
Jul 20 20:43:38.286882 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:43:38.286853    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9152e2fb_1e34_4256_b046_b28df9ab3b7c.slice/crio-29d4a0797bcfcae7f9a8c0a4e876948359c56fb7900d17c39f744bfcbb357eef WatchSource:0}: Error finding container 29d4a0797bcfcae7f9a8c0a4e876948359c56fb7900d17c39f744bfcbb357eef: Status 404 returned error can't find the container with id 29d4a0797bcfcae7f9a8c0a4e876948359c56fb7900d17c39f744bfcbb357eef
Jul 20 20:43:39.118838 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:39.118803    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5" event={"ID":"9152e2fb-1e34-4256-b046-b28df9ab3b7c","Type":"ContainerStarted","Data":"6f90ad41c536214464e0378f7a92372c8f6759d21f115ba3525a1fe8204e27ce"}
Jul 20 20:43:39.118838 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:39.118841    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5" event={"ID":"9152e2fb-1e34-4256-b046-b28df9ab3b7c","Type":"ContainerStarted","Data":"29d4a0797bcfcae7f9a8c0a4e876948359c56fb7900d17c39f744bfcbb357eef"}
Jul 20 20:43:39.137048 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:39.136999    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5" podStartSLOduration=2.136985276 podStartE2EDuration="2.136985276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:43:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:43:39.136382459 +0000 UTC m=+2834.400452505" watchObservedRunningTime="2026-07-20 20:43:39.136985276 +0000 UTC m=+2834.401055318"
Jul 20 20:43:48.146463 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:48.146419    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:48.146946 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:48.146483    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:48.159084 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:48.159060    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:43:48.170544 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:43:48.170523    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:44:27.560319 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:27.560234    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"]
Jul 20 20:44:28.440402 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:28.440367    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"]
Jul 20 20:44:42.579753 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:42.579704    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm" podUID="5e7c6c28-059b-4b48-8c72-938466e1acb9" containerName="main" containerID="cri-o://4ac355d4e3fef973cbc865faadd04470aca26940ddcb8eeba72b87fa39032052" gracePeriod=45
Jul 20 20:44:42.833597 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:42.833531    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:44:42.935341 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:42.935307    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-dshm\") pod \"5e7c6c28-059b-4b48-8c72-938466e1acb9\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") "
Jul 20 20:44:42.935521 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:42.935353    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-tmp-dir\") pod \"5e7c6c28-059b-4b48-8c72-938466e1acb9\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") "
Jul 20 20:44:42.935521 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:42.935387    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e7c6c28-059b-4b48-8c72-938466e1acb9-tls-certs\") pod \"5e7c6c28-059b-4b48-8c72-938466e1acb9\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") "
Jul 20 20:44:42.935521 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:42.935409    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-home\") pod \"5e7c6c28-059b-4b48-8c72-938466e1acb9\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") "
Jul 20 20:44:42.935521 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:42.935477    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-model-cache\") pod \"5e7c6c28-059b-4b48-8c72-938466e1acb9\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") "
Jul 20 20:44:42.935736 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:42.935522    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-287tk\" (UniqueName: \"kubernetes.io/projected/5e7c6c28-059b-4b48-8c72-938466e1acb9-kube-api-access-287tk\") pod \"5e7c6c28-059b-4b48-8c72-938466e1acb9\" (UID: \"5e7c6c28-059b-4b48-8c72-938466e1acb9\") "
Jul 20 20:44:42.935736 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:42.935694    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5e7c6c28-059b-4b48-8c72-938466e1acb9" (UID: "5e7c6c28-059b-4b48-8c72-938466e1acb9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:44:42.935736 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:42.935709    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-home" (OuterVolumeSpecName: "home") pod "5e7c6c28-059b-4b48-8c72-938466e1acb9" (UID: "5e7c6c28-059b-4b48-8c72-938466e1acb9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:44:42.935877 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:42.935794    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-model-cache" (OuterVolumeSpecName: "model-cache") pod "5e7c6c28-059b-4b48-8c72-938466e1acb9" (UID: "5e7c6c28-059b-4b48-8c72-938466e1acb9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:44:42.935877 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:42.935844    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-tmp-dir\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:44:42.935877 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:42.935859    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-home\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:44:42.937532 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:42.937513    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-dshm" (OuterVolumeSpecName: "dshm") pod "5e7c6c28-059b-4b48-8c72-938466e1acb9" (UID: "5e7c6c28-059b-4b48-8c72-938466e1acb9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:44:42.937640 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:42.937534    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e7c6c28-059b-4b48-8c72-938466e1acb9-kube-api-access-287tk" (OuterVolumeSpecName: "kube-api-access-287tk") pod "5e7c6c28-059b-4b48-8c72-938466e1acb9" (UID: "5e7c6c28-059b-4b48-8c72-938466e1acb9"). InnerVolumeSpecName "kube-api-access-287tk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:44:42.937640 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:42.937605    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e7c6c28-059b-4b48-8c72-938466e1acb9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5e7c6c28-059b-4b48-8c72-938466e1acb9" (UID: "5e7c6c28-059b-4b48-8c72-938466e1acb9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 20:44:43.036949 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.036888    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-287tk\" (UniqueName: \"kubernetes.io/projected/5e7c6c28-059b-4b48-8c72-938466e1acb9-kube-api-access-287tk\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:44:43.036949 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.036944    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-dshm\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:44:43.036949 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.036953    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e7c6c28-059b-4b48-8c72-938466e1acb9-tls-certs\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:44:43.036949 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.036988    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e7c6c28-059b-4b48-8c72-938466e1acb9-model-cache\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:44:43.405540 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.405506    2580 generic.go:356] "Generic (PLEG): container finished" podID="5e7c6c28-059b-4b48-8c72-938466e1acb9" containerID="4ac355d4e3fef973cbc865faadd04470aca26940ddcb8eeba72b87fa39032052" exitCode=0
Jul 20 20:44:43.405717 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.405548    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"
Jul 20 20:44:43.405717 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.405547    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm" event={"ID":"5e7c6c28-059b-4b48-8c72-938466e1acb9","Type":"ContainerDied","Data":"4ac355d4e3fef973cbc865faadd04470aca26940ddcb8eeba72b87fa39032052"}
Jul 20 20:44:43.405717 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.405667    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm" event={"ID":"5e7c6c28-059b-4b48-8c72-938466e1acb9","Type":"ContainerDied","Data":"be246ee0934dfb736db51a4b097d290c75126dcedb73ba86c72f8baca2f762c0"}
Jul 20 20:44:43.405717 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.405687    2580 scope.go:117] "RemoveContainer" containerID="4ac355d4e3fef973cbc865faadd04470aca26940ddcb8eeba72b87fa39032052"
Jul 20 20:44:43.413759 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.413743    2580 scope.go:117] "RemoveContainer" containerID="4ac355d4e3fef973cbc865faadd04470aca26940ddcb8eeba72b87fa39032052"
Jul 20 20:44:43.414034 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:44:43.414008    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ac355d4e3fef973cbc865faadd04470aca26940ddcb8eeba72b87fa39032052\": container with ID starting with 4ac355d4e3fef973cbc865faadd04470aca26940ddcb8eeba72b87fa39032052 not found: ID does not exist" containerID="4ac355d4e3fef973cbc865faadd04470aca26940ddcb8eeba72b87fa39032052"
Jul 20 20:44:43.414095 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.414044    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ac355d4e3fef973cbc865faadd04470aca26940ddcb8eeba72b87fa39032052"} err="failed to get container status \"4ac355d4e3fef973cbc865faadd04470aca26940ddcb8eeba72b87fa39032052\": rpc error: code = NotFound desc = could not find container \"4ac355d4e3fef973cbc865faadd04470aca26940ddcb8eeba72b87fa39032052\": container with ID starting with 4ac355d4e3fef973cbc865faadd04470aca26940ddcb8eeba72b87fa39032052 not found: ID does not exist"
Jul 20 20:44:43.421752 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.421729    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"]
Jul 20 20:44:43.425354 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.425334    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b15c6d/rate-test-model-b15c6d-kserve-6dcdb75f59-c7htm"]
Jul 20 20:44:43.463361 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.463256    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5" podUID="9152e2fb-1e34-4256-b046-b28df9ab3b7c" containerName="main" containerID="cri-o://6f90ad41c536214464e0378f7a92372c8f6759d21f115ba3525a1fe8204e27ce" gracePeriod=45
Jul 20 20:44:43.713268 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.713243    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:44:43.844503 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.844467    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9152e2fb-1e34-4256-b046-b28df9ab3b7c-tls-certs\") pod \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") "
Jul 20 20:44:43.844702 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.844541    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-model-cache\") pod \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") "
Jul 20 20:44:43.844702 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.844568    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-dshm\") pod \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") "
Jul 20 20:44:43.844702 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.844600    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fq98p\" (UniqueName: \"kubernetes.io/projected/9152e2fb-1e34-4256-b046-b28df9ab3b7c-kube-api-access-fq98p\") pod \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") "
Jul 20 20:44:43.844702 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.844629    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-tmp-dir\") pod \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") "
Jul 20 20:44:43.844702 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.844649    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-home\") pod \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\" (UID: \"9152e2fb-1e34-4256-b046-b28df9ab3b7c\") "
Jul 20 20:44:43.845019 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.844858    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9152e2fb-1e34-4256-b046-b28df9ab3b7c" (UID: "9152e2fb-1e34-4256-b046-b28df9ab3b7c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:44:43.845019 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.844875    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-model-cache" (OuterVolumeSpecName: "model-cache") pod "9152e2fb-1e34-4256-b046-b28df9ab3b7c" (UID: "9152e2fb-1e34-4256-b046-b28df9ab3b7c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:44:43.845091 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.845013    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-home" (OuterVolumeSpecName: "home") pod "9152e2fb-1e34-4256-b046-b28df9ab3b7c" (UID: "9152e2fb-1e34-4256-b046-b28df9ab3b7c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:44:43.846784 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.846756    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9152e2fb-1e34-4256-b046-b28df9ab3b7c-kube-api-access-fq98p" (OuterVolumeSpecName: "kube-api-access-fq98p") pod "9152e2fb-1e34-4256-b046-b28df9ab3b7c" (UID: "9152e2fb-1e34-4256-b046-b28df9ab3b7c"). InnerVolumeSpecName "kube-api-access-fq98p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:44:43.846784 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.846768    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9152e2fb-1e34-4256-b046-b28df9ab3b7c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9152e2fb-1e34-4256-b046-b28df9ab3b7c" (UID: "9152e2fb-1e34-4256-b046-b28df9ab3b7c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 20:44:43.846784 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.846773    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-dshm" (OuterVolumeSpecName: "dshm") pod "9152e2fb-1e34-4256-b046-b28df9ab3b7c" (UID: "9152e2fb-1e34-4256-b046-b28df9ab3b7c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:44:43.946027 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.945992    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9152e2fb-1e34-4256-b046-b28df9ab3b7c-tls-certs\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:44:43.946027 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.946023    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-model-cache\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:44:43.946027 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.946031    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-dshm\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:44:43.946271 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.946043    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fq98p\" (UniqueName: \"kubernetes.io/projected/9152e2fb-1e34-4256-b046-b28df9ab3b7c-kube-api-access-fq98p\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:44:43.946271 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.946051    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-tmp-dir\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:44:43.946271 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:43.946063    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9152e2fb-1e34-4256-b046-b28df9ab3b7c-home\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:44:44.411510 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:44.411470    2580 generic.go:356] "Generic (PLEG): container finished" podID="9152e2fb-1e34-4256-b046-b28df9ab3b7c" containerID="6f90ad41c536214464e0378f7a92372c8f6759d21f115ba3525a1fe8204e27ce" exitCode=0
Jul 20 20:44:44.411698 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:44.411520    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"
Jul 20 20:44:44.411698 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:44.411554    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5" event={"ID":"9152e2fb-1e34-4256-b046-b28df9ab3b7c","Type":"ContainerDied","Data":"6f90ad41c536214464e0378f7a92372c8f6759d21f115ba3525a1fe8204e27ce"}
Jul 20 20:44:44.411698 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:44.411593    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5" event={"ID":"9152e2fb-1e34-4256-b046-b28df9ab3b7c","Type":"ContainerDied","Data":"29d4a0797bcfcae7f9a8c0a4e876948359c56fb7900d17c39f744bfcbb357eef"}
Jul 20 20:44:44.411698 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:44.411612    2580 scope.go:117] "RemoveContainer" containerID="6f90ad41c536214464e0378f7a92372c8f6759d21f115ba3525a1fe8204e27ce"
Jul 20 20:44:44.420009 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:44.419982    2580 scope.go:117] "RemoveContainer" containerID="6f90ad41c536214464e0378f7a92372c8f6759d21f115ba3525a1fe8204e27ce"
Jul 20 20:44:44.420288 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:44:44.420270    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f90ad41c536214464e0378f7a92372c8f6759d21f115ba3525a1fe8204e27ce\": container with ID starting with 6f90ad41c536214464e0378f7a92372c8f6759d21f115ba3525a1fe8204e27ce not found: ID does not exist" containerID="6f90ad41c536214464e0378f7a92372c8f6759d21f115ba3525a1fe8204e27ce"
Jul 20 20:44:44.420354 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:44.420295    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f90ad41c536214464e0378f7a92372c8f6759d21f115ba3525a1fe8204e27ce"} err="failed to get container status \"6f90ad41c536214464e0378f7a92372c8f6759d21f115ba3525a1fe8204e27ce\": rpc error: code = NotFound desc = could not find container \"6f90ad41c536214464e0378f7a92372c8f6759d21f115ba3525a1fe8204e27ce\": container with ID starting with 6f90ad41c536214464e0378f7a92372c8f6759d21f115ba3525a1fe8204e27ce not found: ID does not exist"
Jul 20 20:44:44.432801 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:44.432772    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"]
Jul 20 20:44:44.436886 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:44.436866    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-40065f/rate-test-model-40065f-kserve-76949f97cf-wvxq5"]
Jul 20 20:44:45.175919 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:45.175887    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e7c6c28-059b-4b48-8c72-938466e1acb9" path="/var/lib/kubelet/pods/5e7c6c28-059b-4b48-8c72-938466e1acb9/volumes"
Jul 20 20:44:45.176310 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:44:45.176279    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9152e2fb-1e34-4256-b046-b28df9ab3b7c" path="/var/lib/kubelet/pods/9152e2fb-1e34-4256-b046-b28df9ab3b7c/volumes"
Jul 20 20:45:00.158490 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:00.158456    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-40065f-29743005-bgjmk"]
Jul 20 20:45:00.158914 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:00.158740    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e7c6c28-059b-4b48-8c72-938466e1acb9" containerName="main"
Jul 20 20:45:00.158914 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:00.158750    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e7c6c28-059b-4b48-8c72-938466e1acb9" containerName="main"
Jul 20 20:45:00.158914 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:00.158770    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9152e2fb-1e34-4256-b046-b28df9ab3b7c" containerName="main"
Jul 20 20:45:00.158914 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:00.158775    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9152e2fb-1e34-4256-b046-b28df9ab3b7c" containerName="main"
Jul 20 20:45:00.158914 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:00.158823    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e7c6c28-059b-4b48-8c72-938466e1acb9" containerName="main"
Jul 20 20:45:00.158914 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:00.158832    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9152e2fb-1e34-4256-b046-b28df9ab3b7c" containerName="main"
Jul 20 20:45:00.162834 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:00.162819    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-40065f-29743005-bgjmk"
Jul 20 20:45:00.166439 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:00.166415    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-628hb\""
Jul 20 20:45:00.172324 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:00.172304    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-40065f-29743005-bgjmk"
Jul 20 20:45:00.173363 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:00.173329    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-40065f-29743005-bgjmk"]
Jul 20 20:45:00.304092 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:00.304067    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-40065f-29743005-bgjmk"]
Jul 20 20:45:00.306307 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:45:00.306277    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c49d32b_b36e_470c_9a8d_690e7aa2ee56.slice/crio-bdf79fa535b4913ae1ea0f802bd56a4153d0f1a1e53bb974f6d79bfbe2346d1a WatchSource:0}: Error finding container bdf79fa535b4913ae1ea0f802bd56a4153d0f1a1e53bb974f6d79bfbe2346d1a: Status 404 returned error can't find the container with id bdf79fa535b4913ae1ea0f802bd56a4153d0f1a1e53bb974f6d79bfbe2346d1a
Jul 20 20:45:00.488047 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:00.488019    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-40065f-29743005-bgjmk" event={"ID":"1c49d32b-b36e-470c-9a8d-690e7aa2ee56","Type":"ContainerStarted","Data":"bdf79fa535b4913ae1ea0f802bd56a4153d0f1a1e53bb974f6d79bfbe2346d1a"}
Jul 20 20:45:01.494295 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:01.494202    2580 generic.go:356] "Generic (PLEG): container finished" podID="1c49d32b-b36e-470c-9a8d-690e7aa2ee56" containerID="09d48d7fd9967d2bdce2f6d4c9f2c34bd8269c9a0e1a883a18b5e21b1c8f89ae" exitCode=0
Jul 20 20:45:01.494658 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:01.494290    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-40065f-29743005-bgjmk" event={"ID":"1c49d32b-b36e-470c-9a8d-690e7aa2ee56","Type":"ContainerDied","Data":"09d48d7fd9967d2bdce2f6d4c9f2c34bd8269c9a0e1a883a18b5e21b1c8f89ae"}
Jul 20 20:45:02.626489 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:02.626465    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-40065f-29743005-bgjmk"
Jul 20 20:45:03.506027 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:03.505931    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-40065f-29743005-bgjmk"
Jul 20 20:45:03.506027 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:03.505946    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-40065f-29743005-bgjmk" event={"ID":"1c49d32b-b36e-470c-9a8d-690e7aa2ee56","Type":"ContainerDied","Data":"bdf79fa535b4913ae1ea0f802bd56a4153d0f1a1e53bb974f6d79bfbe2346d1a"}
Jul 20 20:45:03.506027 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:03.506025    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bdf79fa535b4913ae1ea0f802bd56a4153d0f1a1e53bb974f6d79bfbe2346d1a"
Jul 20 20:45:09.716274 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:09.716238    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"]
Jul 20 20:45:09.717377 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:09.717178    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67" podUID="f938e0df-8655-4de8-940d-6bcf9b227275" containerName="istio-proxy" containerID="cri-o://04e2cd3d144fae9940cf949aca86d05a9fc95ddcd6c6ff17505939676948cac7" gracePeriod=30
Jul 20 20:45:10.076969 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:10.076917    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-40065f-xgjvs"]
Jul 20 20:45:10.077238 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:10.077226    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c49d32b-b36e-470c-9a8d-690e7aa2ee56" containerName="cleanup"
Jul 20 20:45:10.077295 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:10.077240    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c49d32b-b36e-470c-9a8d-690e7aa2ee56" containerName="cleanup"
Jul 20 20:45:10.077333 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:10.077302    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c49d32b-b36e-470c-9a8d-690e7aa2ee56" containerName="cleanup"
Jul 20 20:45:10.080156 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:10.080141    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-40065f-xgjvs"
Jul 20 20:45:10.082834 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:10.082810    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 20:45:10.082951 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:10.082861    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-628hb\""
Jul 20 20:45:10.087790 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:10.087644    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-40065f-xgjvs"]
Jul 20 20:45:10.166493 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:10.166454    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1694c98-f876-48a8-998a-db696307f916-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-40065f-xgjvs\" (UID: \"e1694c98-f876-48a8-998a-db696307f916\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-40065f-xgjvs"
Jul 20 20:45:10.267954 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:10.267917    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1694c98-f876-48a8-998a-db696307f916-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-40065f-xgjvs\" (UID: \"e1694c98-f876-48a8-998a-db696307f916\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-40065f-xgjvs"
Jul 20 20:45:10.268565 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:10.268545    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1694c98-f876-48a8-998a-db696307f916-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-40065f-xgjvs\" (UID: \"e1694c98-f876-48a8-998a-db696307f916\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-40065f-xgjvs"
Jul 20 20:45:10.391669 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:10.391574    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-40065f-xgjvs"
Jul 20 20:45:10.522019 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:10.521992    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-40065f-xgjvs"]
Jul 20 20:45:10.524613 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:45:10.524589    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1694c98_f876_48a8_998a_db696307f916.slice/crio-ec1c2e9dc92c37588af11a0497cd2b74f0f0a0bdc011456bfe06dd6642a00c4b WatchSource:0}: Error finding container ec1c2e9dc92c37588af11a0497cd2b74f0f0a0bdc011456bfe06dd6642a00c4b: Status 404 returned error can't find the container with id ec1c2e9dc92c37588af11a0497cd2b74f0f0a0bdc011456bfe06dd6642a00c4b
Jul 20 20:45:10.537782 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:10.537755    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-40065f-xgjvs" event={"ID":"e1694c98-f876-48a8-998a-db696307f916","Type":"ContainerStarted","Data":"ec1c2e9dc92c37588af11a0497cd2b74f0f0a0bdc011456bfe06dd6642a00c4b"}
Jul 20 20:45:11.544540 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:11.544447    2580 generic.go:356] "Generic (PLEG): container finished" podID="e1694c98-f876-48a8-998a-db696307f916" containerID="8cb187c89bfd8b56590f4295ca45abcff4aa4205d4e05895a15bdd46752089af" exitCode=0
Jul 20 20:45:11.544540 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:11.544516    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-40065f-xgjvs" event={"ID":"e1694c98-f876-48a8-998a-db696307f916","Type":"ContainerDied","Data":"8cb187c89bfd8b56590f4295ca45abcff4aa4205d4e05895a15bdd46752089af"}
Jul 20 20:45:12.670769 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:12.670742    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-40065f-xgjvs"
Jul 20 20:45:12.789334 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:12.789292    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1694c98-f876-48a8-998a-db696307f916-maas-api-service-ca\") pod \"e1694c98-f876-48a8-998a-db696307f916\" (UID: \"e1694c98-f876-48a8-998a-db696307f916\") "
Jul 20 20:45:12.789678 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:12.789659    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1694c98-f876-48a8-998a-db696307f916-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e1694c98-f876-48a8-998a-db696307f916" (UID: "e1694c98-f876-48a8-998a-db696307f916"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 20:45:12.889897 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:12.889825    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1694c98-f876-48a8-998a-db696307f916-maas-api-service-ca\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:45:13.554307 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:13.554280    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-40065f-xgjvs"
Jul 20 20:45:13.554458 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:13.554312    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-40065f-xgjvs" event={"ID":"e1694c98-f876-48a8-998a-db696307f916","Type":"ContainerDied","Data":"ec1c2e9dc92c37588af11a0497cd2b74f0f0a0bdc011456bfe06dd6642a00c4b"}
Jul 20 20:45:13.554458 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:13.554361    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec1c2e9dc92c37588af11a0497cd2b74f0f0a0bdc011456bfe06dd6642a00c4b"
Jul 20 20:45:14.811887 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:14.811851    2580 patch_prober.go:28] interesting pod/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 20 20:45:14.812278 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:14.811911    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67" podUID="f938e0df-8655-4de8-940d-6bcf9b227275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 20 20:45:14.966728 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:14.966699    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:45:15.108658 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.108611    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-istio-envoy\") pod \"f938e0df-8655-4de8-940d-6bcf9b227275\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") "
Jul 20 20:45:15.108658 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.108670    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-credential-socket\") pod \"f938e0df-8655-4de8-940d-6bcf9b227275\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") "
Jul 20 20:45:15.108902 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.108710    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-istio-data\") pod \"f938e0df-8655-4de8-940d-6bcf9b227275\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") "
Jul 20 20:45:15.108902 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.108732    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f938e0df-8655-4de8-940d-6bcf9b227275-istio-podinfo\") pod \"f938e0df-8655-4de8-940d-6bcf9b227275\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") "
Jul 20 20:45:15.108902 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.108757    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-workload-socket\") pod \"f938e0df-8655-4de8-940d-6bcf9b227275\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") "
Jul 20 20:45:15.108902 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.108791    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-workload-certs\") pod \"f938e0df-8655-4de8-940d-6bcf9b227275\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") "
Jul 20 20:45:15.108902 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.108819    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f938e0df-8655-4de8-940d-6bcf9b227275-istiod-ca-cert\") pod \"f938e0df-8655-4de8-940d-6bcf9b227275\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") "
Jul 20 20:45:15.108902 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.108844    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f938e0df-8655-4de8-940d-6bcf9b227275-istio-token\") pod \"f938e0df-8655-4de8-940d-6bcf9b227275\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") "
Jul 20 20:45:15.108902 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.108900    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpxdx\" (UniqueName: \"kubernetes.io/projected/f938e0df-8655-4de8-940d-6bcf9b227275-kube-api-access-zpxdx\") pod \"f938e0df-8655-4de8-940d-6bcf9b227275\" (UID: \"f938e0df-8655-4de8-940d-6bcf9b227275\") "
Jul 20 20:45:15.109295 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.109087    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f938e0df-8655-4de8-940d-6bcf9b227275" (UID: "f938e0df-8655-4de8-940d-6bcf9b227275"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:45:15.109295 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.109109    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f938e0df-8655-4de8-940d-6bcf9b227275" (UID: "f938e0df-8655-4de8-940d-6bcf9b227275"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:45:15.109295 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.109150    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f938e0df-8655-4de8-940d-6bcf9b227275" (UID: "f938e0df-8655-4de8-940d-6bcf9b227275"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:45:15.109295 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.109291    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-credential-socket\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:45:15.109503 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.109312    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-workload-socket\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:45:15.109503 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.109331    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-workload-certs\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:45:15.109751 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.109724    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-istio-data" (OuterVolumeSpecName: "istio-data") pod "f938e0df-8655-4de8-940d-6bcf9b227275" (UID: "f938e0df-8655-4de8-940d-6bcf9b227275"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:45:15.110058 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.110028    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f938e0df-8655-4de8-940d-6bcf9b227275-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f938e0df-8655-4de8-940d-6bcf9b227275" (UID: "f938e0df-8655-4de8-940d-6bcf9b227275"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 20:45:15.111027 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.111000    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f938e0df-8655-4de8-940d-6bcf9b227275" (UID: "f938e0df-8655-4de8-940d-6bcf9b227275"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 20:45:15.111144 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.111039    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f938e0df-8655-4de8-940d-6bcf9b227275-istio-token" (OuterVolumeSpecName: "istio-token") pod "f938e0df-8655-4de8-940d-6bcf9b227275" (UID: "f938e0df-8655-4de8-940d-6bcf9b227275"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:45:15.111144 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.111043    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f938e0df-8655-4de8-940d-6bcf9b227275-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f938e0df-8655-4de8-940d-6bcf9b227275" (UID: "f938e0df-8655-4de8-940d-6bcf9b227275"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 20:45:15.111144 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.111128    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f938e0df-8655-4de8-940d-6bcf9b227275-kube-api-access-zpxdx" (OuterVolumeSpecName: "kube-api-access-zpxdx") pod "f938e0df-8655-4de8-940d-6bcf9b227275" (UID: "f938e0df-8655-4de8-940d-6bcf9b227275"). InnerVolumeSpecName "kube-api-access-zpxdx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 20:45:15.210165 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.210131    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpxdx\" (UniqueName: \"kubernetes.io/projected/f938e0df-8655-4de8-940d-6bcf9b227275-kube-api-access-zpxdx\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:45:15.210165 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.210162    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-istio-envoy\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:45:15.210165 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.210173    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f938e0df-8655-4de8-940d-6bcf9b227275-istio-data\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:45:15.210380 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.210181    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f938e0df-8655-4de8-940d-6bcf9b227275-istio-podinfo\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:45:15.210380 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.210189    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f938e0df-8655-4de8-940d-6bcf9b227275-istiod-ca-cert\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:45:15.210380 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.210197    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f938e0df-8655-4de8-940d-6bcf9b227275-istio-token\") on node \"ip-10-0-143-191.ec2.internal\" DevicePath \"\""
Jul 20 20:45:15.565141 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.565101    2580 generic.go:356] "Generic (PLEG): container finished" podID="f938e0df-8655-4de8-940d-6bcf9b227275" containerID="04e2cd3d144fae9940cf949aca86d05a9fc95ddcd6c6ff17505939676948cac7" exitCode=0
Jul 20 20:45:15.565297 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.565150    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"
Jul 20 20:45:15.565297 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.565177    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67" event={"ID":"f938e0df-8655-4de8-940d-6bcf9b227275","Type":"ContainerDied","Data":"04e2cd3d144fae9940cf949aca86d05a9fc95ddcd6c6ff17505939676948cac7"}
Jul 20 20:45:15.565297 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.565232    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67" event={"ID":"f938e0df-8655-4de8-940d-6bcf9b227275","Type":"ContainerDied","Data":"5f2b98837d79639950152c00bb4cfbe32beb6badefd8da0b0c447a51083ee6c9"}
Jul 20 20:45:15.565297 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.565253    2580 scope.go:117] "RemoveContainer" containerID="04e2cd3d144fae9940cf949aca86d05a9fc95ddcd6c6ff17505939676948cac7"
Jul 20 20:45:15.573583 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.573565    2580 scope.go:117] "RemoveContainer" containerID="f47f279000f507b6c264ae50ae91f5c11680b9b61ecc80c5a97d0fb9a2630485"
Jul 20 20:45:15.581232 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.581210    2580 scope.go:117] "RemoveContainer" containerID="04e2cd3d144fae9940cf949aca86d05a9fc95ddcd6c6ff17505939676948cac7"
Jul 20 20:45:15.581508 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:45:15.581487    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04e2cd3d144fae9940cf949aca86d05a9fc95ddcd6c6ff17505939676948cac7\": container with ID starting with 04e2cd3d144fae9940cf949aca86d05a9fc95ddcd6c6ff17505939676948cac7 not found: ID does not exist" containerID="04e2cd3d144fae9940cf949aca86d05a9fc95ddcd6c6ff17505939676948cac7"
Jul 20 20:45:15.581576 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.581521    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04e2cd3d144fae9940cf949aca86d05a9fc95ddcd6c6ff17505939676948cac7"} err="failed to get container status \"04e2cd3d144fae9940cf949aca86d05a9fc95ddcd6c6ff17505939676948cac7\": rpc error: code = NotFound desc = could not find container \"04e2cd3d144fae9940cf949aca86d05a9fc95ddcd6c6ff17505939676948cac7\": container with ID starting with 04e2cd3d144fae9940cf949aca86d05a9fc95ddcd6c6ff17505939676948cac7 not found: ID does not exist"
Jul 20 20:45:15.581576 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.581548    2580 scope.go:117] "RemoveContainer" containerID="f47f279000f507b6c264ae50ae91f5c11680b9b61ecc80c5a97d0fb9a2630485"
Jul 20 20:45:15.581812 ip-10-0-143-191 kubenswrapper[2580]: E0720 20:45:15.581795    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f47f279000f507b6c264ae50ae91f5c11680b9b61ecc80c5a97d0fb9a2630485\": container with ID starting with f47f279000f507b6c264ae50ae91f5c11680b9b61ecc80c5a97d0fb9a2630485 not found: ID does not exist" containerID="f47f279000f507b6c264ae50ae91f5c11680b9b61ecc80c5a97d0fb9a2630485"
Jul 20 20:45:15.581856 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.581819    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f47f279000f507b6c264ae50ae91f5c11680b9b61ecc80c5a97d0fb9a2630485"} err="failed to get container status \"f47f279000f507b6c264ae50ae91f5c11680b9b61ecc80c5a97d0fb9a2630485\": rpc error: code = NotFound desc = could not find container \"f47f279000f507b6c264ae50ae91f5c11680b9b61ecc80c5a97d0fb9a2630485\": container with ID starting with f47f279000f507b6c264ae50ae91f5c11680b9b61ecc80c5a97d0fb9a2630485 not found: ID does not exist"
Jul 20 20:45:15.585345 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.585324    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"]
Jul 20 20:45:15.590113 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:15.590082    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-b15c6d-openshift-default-848bbdb4bc-lpp67"]
Jul 20 20:45:17.175369 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:17.175334    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f938e0df-8655-4de8-940d-6bcf9b227275" path="/var/lib/kubelet/pods/f938e0df-8655-4de8-940d-6bcf9b227275/volumes"
Jul 20 20:45:20.739747 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:20.739705    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-40065f-29743005-bgjmk"]
Jul 20 20:45:20.744201 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:20.744174    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-40065f-29743005-bgjmk"]
Jul 20 20:45:21.175893 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:21.175855    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c49d32b-b36e-470c-9a8d-690e7aa2ee56" path="/var/lib/kubelet/pods/1c49d32b-b36e-470c-9a8d-690e7aa2ee56/volumes"
Jul 20 20:45:25.246498 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:25.246464    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-40065f-xgjvs"]
Jul 20 20:45:25.252013 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:25.251990    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-40065f-xgjvs"]
Jul 20 20:45:27.175752 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:27.175714    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1694c98-f876-48a8-998a-db696307f916" path="/var/lib/kubelet/pods/e1694c98-f876-48a8-998a-db696307f916/volumes"
Jul 20 20:45:32.720182 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:32.720154    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-58cd897b9f-2dwkq_e4bf2449-c483-4a6b-9f03-1f6949ef6056/authorino/0.log"
Jul 20 20:45:38.462487 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:38.462454    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-77bdf94859-xg8jz_414f1fc4-6669-492b-9ee8-893b2088c3f0/manager/0.log"
Jul 20 20:45:38.704358 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:38.704331    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hkvfk_2c339348-9029-4d9b-854b-a88e192e3266/manager/0.log"
Jul 20 20:45:38.943547 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:38.943516    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77cb6d7555-kl2q9_dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65/copy-manifests/0.log"
Jul 20 20:45:38.953232 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:38.953210    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77cb6d7555-kl2q9_dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65/manager/0.log"
Jul 20 20:45:39.664328 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:39.664293    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29742990-zqbmp_f0d3ba47-4119-4152-9aad-8f7780629cc6/cleanup/0.log"
Jul 20 20:45:41.345870 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:41.345842    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-58cd897b9f-2dwkq_e4bf2449-c483-4a6b-9f03-1f6949ef6056/authorino/0.log"
Jul 20 20:45:41.741488 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:41.741409    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2pkv2_bdf41e73-0a8c-4200-a3d9-5aa7afac963b/kuadrant-console-plugin/0.log"
Jul 20 20:45:42.848351 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:42.848319    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66fb5dc8ff-pmhvf_db75232b-3ac5-48cc-9aea-3b1ec3c1c551/kube-auth-proxy/0.log"
Jul 20 20:45:43.912981 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:43.912936    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-jjmtz_bd2b0998-57e0-42a1-9bf5-ec2808f01eef/main/0.log"
Jul 20 20:45:44.363108 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:44.363078    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-679wt_a555a3f0-e824-4a93-9946-8ffbdf4e76b7/main/0.log"
Jul 20 20:45:50.183557 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.183480    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7mjff/must-gather-stjwl"]
Jul 20 20:45:50.183915 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.183790    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f938e0df-8655-4de8-940d-6bcf9b227275" containerName="istio-proxy"
Jul 20 20:45:50.183915 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.183802    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f938e0df-8655-4de8-940d-6bcf9b227275" containerName="istio-proxy"
Jul 20 20:45:50.183915 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.183821    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1694c98-f876-48a8-998a-db696307f916" containerName="revoke-keys"
Jul 20 20:45:50.183915 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.183828    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1694c98-f876-48a8-998a-db696307f916" containerName="revoke-keys"
Jul 20 20:45:50.183915 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.183835    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f938e0df-8655-4de8-940d-6bcf9b227275" containerName="istio-proxy"
Jul 20 20:45:50.183915 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.183840    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f938e0df-8655-4de8-940d-6bcf9b227275" containerName="istio-proxy"
Jul 20 20:45:50.183915 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.183884    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1694c98-f876-48a8-998a-db696307f916" containerName="revoke-keys"
Jul 20 20:45:50.183915 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.183893    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f938e0df-8655-4de8-940d-6bcf9b227275" containerName="istio-proxy"
Jul 20 20:45:50.184188 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.183998    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f938e0df-8655-4de8-940d-6bcf9b227275" containerName="istio-proxy"
Jul 20 20:45:50.187005 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.186988    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7mjff/must-gather-stjwl"
Jul 20 20:45:50.190112 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.190095    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7mjff\"/\"default-dockercfg-frpwq\""
Jul 20 20:45:50.191285 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.191263    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7mjff\"/\"openshift-service-ca.crt\""
Jul 20 20:45:50.191405 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.191264    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7mjff\"/\"kube-root-ca.crt\""
Jul 20 20:45:50.196896 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.196869    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7mjff/must-gather-stjwl"]
Jul 20 20:45:50.292427 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.292391    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v582f\" (UniqueName: \"kubernetes.io/projected/990ca146-aeb5-43bc-9755-f157ea1d3285-kube-api-access-v582f\") pod \"must-gather-stjwl\" (UID: \"990ca146-aeb5-43bc-9755-f157ea1d3285\") " pod="openshift-must-gather-7mjff/must-gather-stjwl"
Jul 20 20:45:50.292427 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.292443    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/990ca146-aeb5-43bc-9755-f157ea1d3285-must-gather-output\") pod \"must-gather-stjwl\" (UID: \"990ca146-aeb5-43bc-9755-f157ea1d3285\") " pod="openshift-must-gather-7mjff/must-gather-stjwl"
Jul 20 20:45:50.393067 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.393029    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v582f\" (UniqueName: \"kubernetes.io/projected/990ca146-aeb5-43bc-9755-f157ea1d3285-kube-api-access-v582f\") pod \"must-gather-stjwl\" (UID: \"990ca146-aeb5-43bc-9755-f157ea1d3285\") " pod="openshift-must-gather-7mjff/must-gather-stjwl"
Jul 20 20:45:50.393238 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.393078    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/990ca146-aeb5-43bc-9755-f157ea1d3285-must-gather-output\") pod \"must-gather-stjwl\" (UID: \"990ca146-aeb5-43bc-9755-f157ea1d3285\") " pod="openshift-must-gather-7mjff/must-gather-stjwl"
Jul 20 20:45:50.393375 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.393359    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/990ca146-aeb5-43bc-9755-f157ea1d3285-must-gather-output\") pod \"must-gather-stjwl\" (UID: \"990ca146-aeb5-43bc-9755-f157ea1d3285\") " pod="openshift-must-gather-7mjff/must-gather-stjwl"
Jul 20 20:45:50.401889 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.401865    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v582f\" (UniqueName: \"kubernetes.io/projected/990ca146-aeb5-43bc-9755-f157ea1d3285-kube-api-access-v582f\") pod \"must-gather-stjwl\" (UID: \"990ca146-aeb5-43bc-9755-f157ea1d3285\") " pod="openshift-must-gather-7mjff/must-gather-stjwl"
Jul 20 20:45:50.496778 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.496682    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7mjff/must-gather-stjwl"
Jul 20 20:45:50.644418 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.644385    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7mjff/must-gather-stjwl"]
Jul 20 20:45:50.647390 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:45:50.647367    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod990ca146_aeb5_43bc_9755_f157ea1d3285.slice/crio-22801adb3ff6a3ad4e1c6d1f480cce22a8d5367acd27169fba972c7fb19afa21 WatchSource:0}: Error finding container 22801adb3ff6a3ad4e1c6d1f480cce22a8d5367acd27169fba972c7fb19afa21: Status 404 returned error can't find the container with id 22801adb3ff6a3ad4e1c6d1f480cce22a8d5367acd27169fba972c7fb19afa21
Jul 20 20:45:50.649283 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.649263    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 20:45:50.710494 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:50.710463    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7mjff/must-gather-stjwl" event={"ID":"990ca146-aeb5-43bc-9755-f157ea1d3285","Type":"ContainerStarted","Data":"22801adb3ff6a3ad4e1c6d1f480cce22a8d5367acd27169fba972c7fb19afa21"}
Jul 20 20:45:52.730149 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:52.729541    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7mjff/must-gather-stjwl" event={"ID":"990ca146-aeb5-43bc-9755-f157ea1d3285","Type":"ContainerStarted","Data":"ac53cbaad609e233de83e0373580e6cfc96b15076df2da268f702014c1ea4204"}
Jul 20 20:45:52.730559 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:52.730173    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7mjff/must-gather-stjwl" event={"ID":"990ca146-aeb5-43bc-9755-f157ea1d3285","Type":"ContainerStarted","Data":"598c2199bed746e4121bba9b8d0aa8daba9672c4bf9e27a363e584b7bb0fbd5c"}
Jul 20 20:45:52.747990 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:52.747918    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7mjff/must-gather-stjwl" podStartSLOduration=1.748493643 podStartE2EDuration="2.747902045s" totalImagesPullingTime="999.408402ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:45:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:45:52.747344032 +0000 UTC m=+2968.011414079" watchObservedRunningTime="2026-07-20 20:45:52.747902045 +0000 UTC m=+2968.011972093"
Jul 20 20:45:53.301314 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:53.301273    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xlc4n_2fbbdeaa-171d-4032-9bd3-7bbd4c009388/global-pull-secret-syncer/0.log"
Jul 20 20:45:53.399518 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:53.399487    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nb8cs_d518f591-565e-4ac1-b5d1-f88311868032/konnectivity-agent/0.log"
Jul 20 20:45:53.501276 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:53.501247    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-191.ec2.internal_504046be7821a23dec7f8227b49cfc17/haproxy/0.log"
Jul 20 20:45:58.491371 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:58.491320    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-58cd897b9f-2dwkq_e4bf2449-c483-4a6b-9f03-1f6949ef6056/authorino/0.log"
Jul 20 20:45:58.582771 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:45:58.582738    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2pkv2_bdf41e73-0a8c-4200-a3d9-5aa7afac963b/kuadrant-console-plugin/0.log"
Jul 20 20:46:00.926910 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:00.926881    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jh4d7_e39ead50-350e-4546-ba14-6e116bb537cb/node-exporter/0.log"
Jul 20 20:46:00.947521 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:00.947474    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jh4d7_e39ead50-350e-4546-ba14-6e116bb537cb/kube-rbac-proxy/0.log"
Jul 20 20:46:00.969886 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:00.969860    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jh4d7_e39ead50-350e-4546-ba14-6e116bb537cb/init-textfile/0.log"
Jul 20 20:46:02.930435 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:02.930394    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm"]
Jul 20 20:46:02.937897 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:02.937868    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm"
Jul 20 20:46:02.942532 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:02.942503    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm"]
Jul 20 20:46:03.014276 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:03.014243    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/142e0b9f-e98b-4a68-accd-be6a8efc323f-lib-modules\") pod \"perf-node-gather-daemonset-pb5pm\" (UID: \"142e0b9f-e98b-4a68-accd-be6a8efc323f\") " pod="openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm"
Jul 20 20:46:03.014628 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:03.014603    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/142e0b9f-e98b-4a68-accd-be6a8efc323f-podres\") pod \"perf-node-gather-daemonset-pb5pm\" (UID: \"142e0b9f-e98b-4a68-accd-be6a8efc323f\") " pod="openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm"
Jul 20 20:46:03.014780 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:03.014765    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/142e0b9f-e98b-4a68-accd-be6a8efc323f-sys\") pod \"perf-node-gather-daemonset-pb5pm\" (UID: \"142e0b9f-e98b-4a68-accd-be6a8efc323f\") " pod="openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm"
Jul 20 20:46:03.015031 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:03.015005    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdh2g\" (UniqueName: \"kubernetes.io/projected/142e0b9f-e98b-4a68-accd-be6a8efc323f-kube-api-access-fdh2g\") pod \"perf-node-gather-daemonset-pb5pm\" (UID: \"142e0b9f-e98b-4a68-accd-be6a8efc323f\") " pod="openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm"
Jul 20 20:46:03.015179 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:03.015163    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/142e0b9f-e98b-4a68-accd-be6a8efc323f-proc\") pod \"perf-node-gather-daemonset-pb5pm\" (UID: \"142e0b9f-e98b-4a68-accd-be6a8efc323f\") " pod="openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm"
Jul 20 20:46:03.116783 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:03.116745    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/142e0b9f-e98b-4a68-accd-be6a8efc323f-sys\") pod \"perf-node-gather-daemonset-pb5pm\" (UID: \"142e0b9f-e98b-4a68-accd-be6a8efc323f\") " pod="openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm"
Jul 20 20:46:03.116997 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:03.116883    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/142e0b9f-e98b-4a68-accd-be6a8efc323f-sys\") pod \"perf-node-gather-daemonset-pb5pm\" (UID: \"142e0b9f-e98b-4a68-accd-be6a8efc323f\") " pod="openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm"
Jul 20 20:46:03.117070 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:03.117009    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdh2g\" (UniqueName: \"kubernetes.io/projected/142e0b9f-e98b-4a68-accd-be6a8efc323f-kube-api-access-fdh2g\") pod \"perf-node-gather-daemonset-pb5pm\" (UID: \"142e0b9f-e98b-4a68-accd-be6a8efc323f\") " pod="openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm"
Jul 20 20:46:03.117339 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:03.117285    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/142e0b9f-e98b-4a68-accd-be6a8efc323f-proc\") pod \"perf-node-gather-daemonset-pb5pm\" (UID: \"142e0b9f-e98b-4a68-accd-be6a8efc323f\") " pod="openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm"
Jul 20 20:46:03.117575 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:03.117552    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/142e0b9f-e98b-4a68-accd-be6a8efc323f-proc\") pod \"perf-node-gather-daemonset-pb5pm\" (UID: \"142e0b9f-e98b-4a68-accd-be6a8efc323f\") " pod="openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm"
Jul 20 20:46:03.117662 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:03.117625    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/142e0b9f-e98b-4a68-accd-be6a8efc323f-lib-modules\") pod \"perf-node-gather-daemonset-pb5pm\" (UID: \"142e0b9f-e98b-4a68-accd-be6a8efc323f\") " pod="openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm"
Jul 20 20:46:03.117734 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:03.117698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/142e0b9f-e98b-4a68-accd-be6a8efc323f-podres\") pod \"perf-node-gather-daemonset-pb5pm\" (UID: \"142e0b9f-e98b-4a68-accd-be6a8efc323f\") " pod="openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm"
Jul 20 20:46:03.118032 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:03.117879    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/142e0b9f-e98b-4a68-accd-be6a8efc323f-lib-modules\") pod \"perf-node-gather-daemonset-pb5pm\" (UID: \"142e0b9f-e98b-4a68-accd-be6a8efc323f\") " pod="openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm"
Jul 20 20:46:03.118032 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:03.117995    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/142e0b9f-e98b-4a68-accd-be6a8efc323f-podres\") pod \"perf-node-gather-daemonset-pb5pm\" (UID: \"142e0b9f-e98b-4a68-accd-be6a8efc323f\") " pod="openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm"
Jul 20 20:46:03.126006 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:03.125982    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdh2g\" (UniqueName: \"kubernetes.io/projected/142e0b9f-e98b-4a68-accd-be6a8efc323f-kube-api-access-fdh2g\") pod \"perf-node-gather-daemonset-pb5pm\" (UID: \"142e0b9f-e98b-4a68-accd-be6a8efc323f\") " pod="openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm"
Jul 20 20:46:03.251249 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:03.251154    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm"
Jul 20 20:46:03.432594 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:03.431832    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm"]
Jul 20 20:46:03.436502 ip-10-0-143-191 kubenswrapper[2580]: W0720 20:46:03.436412    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod142e0b9f_e98b_4a68_accd_be6a8efc323f.slice/crio-4b72831854f986456891e48e1076222de3604e978954672d8dbaffe2ac8f76da WatchSource:0}: Error finding container 4b72831854f986456891e48e1076222de3604e978954672d8dbaffe2ac8f76da: Status 404 returned error can't find the container with id 4b72831854f986456891e48e1076222de3604e978954672d8dbaffe2ac8f76da
Jul 20 20:46:03.807092 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:03.807045    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm" event={"ID":"142e0b9f-e98b-4a68-accd-be6a8efc323f","Type":"ContainerStarted","Data":"4b40f01fd3d42dbb51611a0dcf88d5c45f27b4ad5e6541cd445f84c24ca4c4f7"}
Jul 20 20:46:03.807288 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:03.807121    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm" event={"ID":"142e0b9f-e98b-4a68-accd-be6a8efc323f","Type":"ContainerStarted","Data":"4b72831854f986456891e48e1076222de3604e978954672d8dbaffe2ac8f76da"}
Jul 20 20:46:03.807933 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:03.807909    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm"
Jul 20 20:46:03.825573 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:03.825467    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm" podStartSLOduration=1.825446464 podStartE2EDuration="1.825446464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 20:46:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:46:03.823890599 +0000 UTC m=+2979.087960650" watchObservedRunningTime="2026-07-20 20:46:03.825446464 +0000 UTC m=+2979.089516513"
Jul 20 20:46:05.676271 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:05.676243    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nn6k7_3b9c6f82-5df3-4fec-b041-a6f3615d7949/dns/0.log"
Jul 20 20:46:05.698164 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:05.698141    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nn6k7_3b9c6f82-5df3-4fec-b041-a6f3615d7949/kube-rbac-proxy/0.log"
Jul 20 20:46:05.787009 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:05.786979    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nngvn_a164b5a7-e049-42a5-8f89-915704f82f68/dns-node-resolver/0.log"
Jul 20 20:46:06.334905 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:06.334875    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-77c574ccb-8757d_903a01f6-5dc1-4d21-a0a1-0a2fac389d1c/registry/0.log"
Jul 20 20:46:06.403496 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:06.403439    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w5555_b4c1d9fe-73ea-4091-9855-b9f6970d176e/node-ca/0.log"
Jul 20 20:46:07.561595 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:07.561568    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66fb5dc8ff-pmhvf_db75232b-3ac5-48cc-9aea-3b1ec3c1c551/kube-auth-proxy/0.log"
Jul 20 20:46:08.481981 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:08.481931    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xvdd8_9ac271c1-f11b-4518-a768-697ed858df99/serve-healthcheck-canary/0.log"
Jul 20 20:46:09.143386 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:09.143344    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mmxqq_038028aa-4904-47b7-8284-0197ad123412/kube-rbac-proxy/0.log"
Jul 20 20:46:09.164434 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:09.164410    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mmxqq_038028aa-4904-47b7-8284-0197ad123412/exporter/0.log"
Jul 20 20:46:09.187026 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:09.187001    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mmxqq_038028aa-4904-47b7-8284-0197ad123412/extractor/0.log"
Jul 20 20:46:10.835502 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:10.835468    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7mjff/perf-node-gather-daemonset-pb5pm"
Jul 20 20:46:11.704724 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:11.704683    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-77bdf94859-xg8jz_414f1fc4-6669-492b-9ee8-893b2088c3f0/manager/0.log"
Jul 20 20:46:11.823144 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:11.823111    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hkvfk_2c339348-9029-4d9b-854b-a88e192e3266/manager/0.log"
Jul 20 20:46:11.899400 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:11.899369    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77cb6d7555-kl2q9_dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65/manager/0.log"
Jul 20 20:46:11.918600 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:11.918568    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77cb6d7555-kl2q9_dbc0ed85-28a8-4278-ae9b-0c2da1ce0c65/copy-manifests/0.log"
Jul 20 20:46:13.468913 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:13.468824    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7f6758cc7d-zs24s_a4c6b53a-c84f-4531-a852-f827f8a38a46/manager/0.log"
Jul 20 20:46:13.547749 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:13.547698    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66fb5dc8ff-pmhvf_db75232b-3ac5-48cc-9aea-3b1ec3c1c551/kube-auth-proxy/0.log"
Jul 20 20:46:15.145972 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:15.145926    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66fb5dc8ff-pmhvf_db75232b-3ac5-48cc-9aea-3b1ec3c1c551/kube-auth-proxy/0.log"
Jul 20 20:46:18.447243 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:18.447212    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-vc57q_ab77f7a5-598d-4e32-be06-93185adc3e76/migrator/0.log"
Jul 20 20:46:18.467762 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:18.467728    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-vc57q_ab77f7a5-598d-4e32-be06-93185adc3e76/graceful-termination/0.log"
Jul 20 20:46:19.938189 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:19.938162    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b94k5_bfaea5fd-9480-4cd0-b133-217c1eb6069a/kube-multus-additional-cni-plugins/0.log"
Jul 20 20:46:19.960206 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:19.960177    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b94k5_bfaea5fd-9480-4cd0-b133-217c1eb6069a/egress-router-binary-copy/0.log"
Jul 20 20:46:19.983070 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:19.983010    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b94k5_bfaea5fd-9480-4cd0-b133-217c1eb6069a/cni-plugins/0.log"
Jul 20 20:46:20.005897 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:20.005871    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b94k5_bfaea5fd-9480-4cd0-b133-217c1eb6069a/bond-cni-plugin/0.log"
Jul 20 20:46:20.025147 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:20.025115    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b94k5_bfaea5fd-9480-4cd0-b133-217c1eb6069a/routeoverride-cni/0.log"
Jul 20 20:46:20.045225 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:20.045201    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b94k5_bfaea5fd-9480-4cd0-b133-217c1eb6069a/whereabouts-cni-bincopy/0.log"
Jul 20 20:46:20.065884 ip-10-0-143-191 kubenswrapper[2580]: I0720 20:46:20.065856    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b94k5_bfaea5fd-9480-4cd0-b133-217c1eb6069a/whereabouts-cni/0.log"