Jul 13 19:19:29.622577 ip-10-0-143-91 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 13 19:19:29.622589 ip-10-0-143-91 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 13 19:19:29.622596 ip-10-0-143-91 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 13 19:19:29.622818 ip-10-0-143-91 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 13 19:19:39.752446 ip-10-0-143-91 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 13 19:19:39.752469 ip-10-0-143-91 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 2376c35104184075ba93bd8b35bdb0cf --
Jul 13 19:21:42.372989 ip-10-0-143-91 systemd[1]: Starting Kubernetes Kubelet...
Jul 13 19:21:42.761473 ip-10-0-143-91 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 19:21:42.761473 ip-10-0-143-91 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 13 19:21:42.761473 ip-10-0-143-91 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 19:21:42.761473 ip-10-0-143-91 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 13 19:21:42.761473 ip-10-0-143-91 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 19:21:42.763218 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.763126    2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 13 19:21:42.768272 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768252    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 19:21:42.768272 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768272    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 19:21:42.768350 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768278    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 19:21:42.768350 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768281    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 19:21:42.768350 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768285    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 19:21:42.768350 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768287    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 19:21:42.768350 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768290    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 19:21:42.768350 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768293    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 19:21:42.768350 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768297    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 19:21:42.768350 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768300    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 19:21:42.768350 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768303    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 19:21:42.768350 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768305    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 19:21:42.768350 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768308    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 19:21:42.768350 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768311    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 19:21:42.768350 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768313    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 19:21:42.768350 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768316    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 19:21:42.768350 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768319    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 19:21:42.768350 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768321    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 19:21:42.768350 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768324    2573 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 19:21:42.768350 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768327    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 19:21:42.768350 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768330    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 19:21:42.768350 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768332    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 19:21:42.768826 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768335    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 19:21:42.768826 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768338    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 19:21:42.768826 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768341    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 19:21:42.768826 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768344    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 19:21:42.768826 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768346    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 19:21:42.768826 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768349    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 19:21:42.768826 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768352    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 19:21:42.768826 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768354    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 19:21:42.768826 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768357    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 19:21:42.768826 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768360    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 19:21:42.768826 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768363    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 19:21:42.768826 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768366    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 19:21:42.768826 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768368    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 19:21:42.768826 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768371    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 19:21:42.768826 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768375    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 19:21:42.768826 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768379    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 19:21:42.768826 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768382    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 19:21:42.768826 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768386    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 19:21:42.768826 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768388    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 19:21:42.768826 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768391    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 19:21:42.769307 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768394    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 19:21:42.769307 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768397    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 19:21:42.769307 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768400    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 19:21:42.769307 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768403    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 19:21:42.769307 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768406    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 19:21:42.769307 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768409    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 19:21:42.769307 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768412    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 19:21:42.769307 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768414    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 19:21:42.769307 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768417    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 19:21:42.769307 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768420    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 19:21:42.769307 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768422    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 19:21:42.769307 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768425    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 19:21:42.769307 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768428    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 19:21:42.769307 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768431    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 19:21:42.769307 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768434    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 19:21:42.769307 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768437    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 19:21:42.769307 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768440    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 19:21:42.769307 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768443    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 19:21:42.769307 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768445    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 19:21:42.769307 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768448    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 19:21:42.769799 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768451    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 19:21:42.769799 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768454    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 19:21:42.769799 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768457    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 19:21:42.769799 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768460    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 19:21:42.769799 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768463    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 19:21:42.769799 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768465    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 19:21:42.769799 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768468    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 19:21:42.769799 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768471    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 19:21:42.769799 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768473    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 19:21:42.769799 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768477    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 19:21:42.769799 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768479    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 19:21:42.769799 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768482    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 19:21:42.769799 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768484    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 19:21:42.769799 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768488    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 19:21:42.769799 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768491    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 19:21:42.769799 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768494    2573 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 19:21:42.769799 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768497    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 19:21:42.769799 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768499    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 19:21:42.769799 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768502    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 19:21:42.770245 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768504    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 19:21:42.770245 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768507    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 19:21:42.770245 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768510    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 19:21:42.770245 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768512    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 19:21:42.770245 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768515    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 19:21:42.770245 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768517    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 19:21:42.770245 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768523    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 19:21:42.770245 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768526    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 19:21:42.770245 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768529    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 19:21:42.770245 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768532    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 19:21:42.770245 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768534    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 19:21:42.770245 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768537    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 19:21:42.770245 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768539    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 19:21:42.770245 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768542    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 19:21:42.770245 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768545    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 19:21:42.770245 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768547    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 19:21:42.770245 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768550    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 19:21:42.770245 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.768553    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 19:21:42.770245 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769317    2573 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 19:21:42.770718 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769324    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 19:21:42.770718 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769328    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 19:21:42.770718 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769331    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 19:21:42.770718 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769334    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 19:21:42.770718 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769338    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 19:21:42.770718 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769341    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 19:21:42.770718 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769344    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 19:21:42.770718 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769347    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 19:21:42.770718 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769350    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 19:21:42.770718 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769352    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 19:21:42.770718 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769355    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 19:21:42.770718 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769358    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 19:21:42.770718 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769361    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 19:21:42.770718 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769363    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 19:21:42.770718 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769366    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 19:21:42.770718 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769370    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 19:21:42.770718 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769374    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 19:21:42.770718 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769377    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 19:21:42.770718 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769380    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 19:21:42.771186 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769384    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 19:21:42.771186 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769387    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 19:21:42.771186 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769390    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 19:21:42.771186 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769393    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 19:21:42.771186 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769396    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 19:21:42.771186 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769398    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 19:21:42.771186 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769401    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 19:21:42.771186 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769404    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 19:21:42.771186 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769406    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 19:21:42.771186 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769409    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 19:21:42.771186 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769412    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 19:21:42.771186 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769414    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 19:21:42.771186 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769417    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 19:21:42.771186 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769419    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 19:21:42.771186 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769422    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 19:21:42.771186 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769424    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 19:21:42.771186 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769427    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 19:21:42.771186 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769429    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 19:21:42.771186 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769432    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 19:21:42.771660 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769435    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 19:21:42.771660 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769438    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 19:21:42.771660 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769440    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 19:21:42.771660 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769443    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 19:21:42.771660 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769447    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 19:21:42.771660 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769451    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 19:21:42.771660 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769454    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 19:21:42.771660 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769456    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 19:21:42.771660 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769459    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 19:21:42.771660 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769462    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 19:21:42.771660 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769464    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 19:21:42.771660 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769467    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 19:21:42.771660 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769470    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 19:21:42.771660 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769473    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 19:21:42.771660 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769476    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 19:21:42.771660 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769478    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 19:21:42.771660 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769481    2573 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 19:21:42.771660 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769484    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 19:21:42.771660 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769488    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 19:21:42.772120 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769491    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 19:21:42.772120 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769494    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 19:21:42.772120 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769496    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 19:21:42.772120 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769499    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 19:21:42.772120 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769501    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 19:21:42.772120 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769504    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 19:21:42.772120 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769506    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 19:21:42.772120 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769509    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 19:21:42.772120 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769512    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 19:21:42.772120 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769514    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 19:21:42.772120 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769517    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 19:21:42.772120 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769520    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 19:21:42.772120 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769523    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 19:21:42.772120 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769526    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 19:21:42.772120 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769529    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 19:21:42.772120 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769533    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 19:21:42.772120 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769535    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 19:21:42.772120 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769538    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 19:21:42.772120 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769541    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 19:21:42.772120 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769543    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 19:21:42.772621 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769546    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 19:21:42.772621 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769548    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 19:21:42.772621 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769551    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 19:21:42.772621 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769553    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 19:21:42.772621 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769556    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 19:21:42.772621 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769558    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 19:21:42.772621 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769561    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 19:21:42.772621 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769564    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 19:21:42.772621 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769567    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 19:21:42.772621 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769571    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 19:21:42.772621 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769574    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 19:21:42.772621 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769577    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 19:21:42.772621 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769580    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 19:21:42.772621 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769583    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 19:21:42.772621 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769586    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 19:21:42.772621 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769589    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 19:21:42.772621 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769592    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 19:21:42.772621 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769595    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 19:21:42.772621 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769598    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 19:21:42.772621 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769600    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 19:21:42.773128 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.769603    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 19:21:42.773128 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770037    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 19:21:42.773128 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770043    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 19:21:42.773128 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770046    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 19:21:42.773128 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770049    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 19:21:42.773128 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770053    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 19:21:42.773128 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770057    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 19:21:42.773128 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770060    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 19:21:42.773128 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770064    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 19:21:42.773128 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770066    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 19:21:42.773128 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770070    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 19:21:42.773128 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770073    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 19:21:42.773128 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770076    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 19:21:42.773128 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770078    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 19:21:42.773128 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770081    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 19:21:42.773128 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770084    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 19:21:42.773128 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770087    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 19:21:42.773128 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770090    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 19:21:42.773128 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770096    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 19:21:42.773584 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770099    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 19:21:42.773584 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770101    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 19:21:42.773584 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770104    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 19:21:42.773584 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770107    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 19:21:42.773584 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770109    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 19:21:42.773584 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770113    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 19:21:42.773584 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770115    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 19:21:42.773584 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770118    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 19:21:42.773584 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770121    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 19:21:42.773584 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770123    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 19:21:42.773584 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770126    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 19:21:42.773584 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770129    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 19:21:42.773584 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770132    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 19:21:42.773584 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770134    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 19:21:42.773584 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770137    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 19:21:42.773584 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770139    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 19:21:42.773584 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770142    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 19:21:42.773584 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770145    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 19:21:42.773584 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770148    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 19:21:42.773584 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770151    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 19:21:42.774107 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770154    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 19:21:42.774107 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770156    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 19:21:42.774107 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770159    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 19:21:42.774107 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770161    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 19:21:42.774107 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770163    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 19:21:42.774107 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770166    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 19:21:42.774107 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770170    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 19:21:42.774107 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770172    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 19:21:42.774107 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770174    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 19:21:42.774107 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770177    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 19:21:42.774107 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770180    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 19:21:42.774107 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770183    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 19:21:42.774107 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770186    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 19:21:42.774107 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770189    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 19:21:42.774107 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770191    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 19:21:42.774107 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770194    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 19:21:42.774107 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770197    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 19:21:42.774107 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770201    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 19:21:42.774107 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770204    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 19:21:42.774107 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770206    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 19:21:42.774595 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770209    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 19:21:42.774595 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770213    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 19:21:42.774595 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770217    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 19:21:42.774595 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770220    2573 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 19:21:42.774595 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770223    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 19:21:42.774595 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770226    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 19:21:42.774595 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770229    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 19:21:42.774595 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770232    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 19:21:42.774595 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770235    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 19:21:42.774595 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770238    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 19:21:42.774595 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770242    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 19:21:42.774595 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770246    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 19:21:42.774595 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770249    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 19:21:42.774595 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770254    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 19:21:42.774595 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770257    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 19:21:42.774595 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770259    2573 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 19:21:42.774595 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770262    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 19:21:42.774595 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770265    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 19:21:42.774595 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770267    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 19:21:42.774595 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770270    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 19:21:42.775099 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770272    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 19:21:42.775099 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770275    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 19:21:42.775099 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770277    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 19:21:42.775099 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770283    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 19:21:42.775099 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770286    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 19:21:42.775099 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770290    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 19:21:42.775099 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770293    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 19:21:42.775099 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770296    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 19:21:42.775099 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770299    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 19:21:42.775099 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770303    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 19:21:42.775099 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770306    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 19:21:42.775099 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770309    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 19:21:42.775099 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770312    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 19:21:42.775099 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770314    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 19:21:42.775099 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770317    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 19:21:42.775099 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770320    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 19:21:42.775099 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770323    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 19:21:42.775099 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770326    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 19:21:42.775099 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770328    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 19:21:42.775099 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770331    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.770333    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770403    2573 flags.go:64] FLAG: --address="0.0.0.0"
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770410    2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770416    2573 flags.go:64] FLAG: --anonymous-auth="true"
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770421    2573 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770428    2573 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770433    2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770439    2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770446    2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770451    2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770454    2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770458    2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770461    2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770464    2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770467    2573 flags.go:64] FLAG: --cgroup-root=""
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770471    2573 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770474    2573 flags.go:64] FLAG: --client-ca-file=""
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770477    2573 flags.go:64] FLAG: --cloud-provider="external"
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770480    2573 flags.go:64] FLAG: --cluster-dns="[]"
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770485    2573 flags.go:64] FLAG: --cluster-domain=""
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770488    2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770492    2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770496    2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 13 19:21:42.775617 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770499    2573 flags.go:64] FLAG: --container-log-max-files="5"
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770503    2573 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770506    2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770510    2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770514    2573 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770517    2573 flags.go:64] FLAG: --contention-profiling="false"
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770520    2573 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770524    2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770527    2573 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770530    2573 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770534    2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770537    2573 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770541    2573 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770543    2573 flags.go:64] FLAG: --enable-load-reader="false"
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770547    2573 flags.go:64] FLAG: --enable-server="true"
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770550    2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770554    2573 flags.go:64] FLAG: --event-burst="100"
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770558    2573 flags.go:64] FLAG: --event-qps="50"
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770561    2573 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770564    2573 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770566    2573 flags.go:64] FLAG: --eviction-hard=""
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770571    2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770574    2573 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770577    2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770581    2573 flags.go:64] FLAG: --eviction-soft=""
Jul 13 19:21:42.776206 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770584    2573 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770587    2573 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770590    2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770593    2573 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770596    2573 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770600    2573 flags.go:64] FLAG: --fail-swap-on="true"
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770603    2573 flags.go:64] FLAG: --feature-gates=""
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770607    2573 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770610    2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770613    2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770616    2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770619    2573 flags.go:64] FLAG: --healthz-port="10248"
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770622    2573 flags.go:64] FLAG: --help="false"
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770625    2573 flags.go:64] FLAG: --hostname-override="ip-10-0-143-91.ec2.internal"
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770628    2573 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770645    2573 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770651    2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770656    2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770660    2573 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770663    2573 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770666    2573 flags.go:64] FLAG: --image-service-endpoint=""
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770669    2573 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770672    2573 flags.go:64] FLAG: --kube-api-burst="100"
Jul 13 19:21:42.776843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770676    2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770679    2573 flags.go:64] FLAG: --kube-api-qps="50"
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770682    2573 flags.go:64] FLAG: --kube-reserved=""
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770685    2573 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770688    2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770691    2573 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770694    2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770697    2573 flags.go:64] FLAG: --lock-file=""
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770701    2573 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770704    2573 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770707    2573 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770713    2573 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770715    2573 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770718    2573 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770722    2573 flags.go:64] FLAG: --logging-format="text"
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770725    2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770729    2573 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770732    2573 flags.go:64] FLAG: --manifest-url=""
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770735    2573 flags.go:64] FLAG: --manifest-url-header=""
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770739    2573 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770742    2573 flags.go:64] FLAG: --max-open-files="1000000"
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770746    2573 flags.go:64] FLAG: --max-pods="110"
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770749    2573 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770752    2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770755    2573 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 13 19:21:42.777414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770758    2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770761    2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770764    2573 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770767    2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770774    2573 flags.go:64] FLAG: --node-status-max-images="50"
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770777    2573 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770780    2573 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770783    2573 flags.go:64] FLAG: --pod-cidr=""
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770787    2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770793    2573 flags.go:64] FLAG: --pod-manifest-path=""
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770796    2573 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770799    2573 flags.go:64] FLAG: --pods-per-core="0"
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770803    2573 flags.go:64] FLAG: --port="10250"
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770806    2573 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770809    2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01214fa5b306531bc"
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770812    2573 flags.go:64] FLAG: --qos-reserved=""
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770816    2573 flags.go:64] FLAG: --read-only-port="10255"
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770819    2573 flags.go:64] FLAG: --register-node="true"
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770822    2573 flags.go:64] FLAG: --register-with-taints=""
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770833    2573 flags.go:64] FLAG: --registry-burst="10"
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770837    2573 flags.go:64] FLAG: --registry-qps="5"
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770841    2573 flags.go:64] FLAG: --reserved-cpus=""
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770844    2573 flags.go:64] FLAG: --reserved-memory=""
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770848    2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770851    2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 13 19:21:42.778041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770854    2573 flags.go:64] FLAG: --rotate-certificates="false"
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770857    2573 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770860    2573 flags.go:64] FLAG: --runonce="false"
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770862    2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770865    2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770868    2573 flags.go:64] FLAG: --seccomp-default="false"
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770871    2573 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770874    2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770877    2573 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770880    2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770883    2573 flags.go:64] FLAG: --storage-driver-password="root"
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770886    2573 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770889    2573 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770892    2573 flags.go:64] FLAG: --storage-driver-user="root"
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770895    2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770898    2573 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770901    2573 flags.go:64] FLAG: --system-cgroups=""
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770904    2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770909    2573 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770912    2573 flags.go:64] FLAG: --tls-cert-file=""
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770915    2573 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770919    2573 flags.go:64] FLAG: --tls-min-version=""
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770922    2573 flags.go:64] FLAG: --tls-private-key-file=""
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770925    2573 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770928    2573 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 13 19:21:42.778652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770931    2573 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 13 19:21:42.779300 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770934    2573 flags.go:64] FLAG: --v="2"
Jul 13 19:21:42.779300 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770938    2573 flags.go:64] FLAG: --version="false"
Jul 13 19:21:42.779300 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770947    2573 flags.go:64] FLAG: --vmodule=""
Jul 13 19:21:42.779300 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770951    2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 13 19:21:42.779300 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.770954    2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 13 19:21:42.779300 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771075    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 19:21:42.779300 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771080    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 19:21:42.779300 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771084    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 19:21:42.779300 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771088    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 19:21:42.779300 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771091    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 19:21:42.779300 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771094    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 19:21:42.779300 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771097    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 19:21:42.779300 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771100    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 19:21:42.779300 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771102    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 19:21:42.779300 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771105    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 19:21:42.779300 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771108    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 19:21:42.779300 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771111    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 19:21:42.779300 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771113    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 19:21:42.779300 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771116    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 19:21:42.779300 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771118    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 19:21:42.779300 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771121    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 19:21:42.779891 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771123    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 19:21:42.779891 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771126    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 19:21:42.779891 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771130    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 19:21:42.779891 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771132    2573 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 19:21:42.779891 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771136    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 19:21:42.779891 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771138    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 19:21:42.779891 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771142    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 19:21:42.779891 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771144    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 19:21:42.779891 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771147    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 19:21:42.779891 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771149    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 19:21:42.779891 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771152    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 19:21:42.779891 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771154    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 19:21:42.779891 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771157    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 19:21:42.779891 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771161    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 19:21:42.779891 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771163    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 19:21:42.779891 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771166    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 19:21:42.779891 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771169    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 19:21:42.779891 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771171    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 19:21:42.779891 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771173    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 19:21:42.779891 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771176    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 19:21:42.779891 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771178    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 19:21:42.780501 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771181    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 19:21:42.780501 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771184    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 19:21:42.780501 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771186    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 19:21:42.780501 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771189    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 19:21:42.780501 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771191    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 19:21:42.780501 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771194    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 19:21:42.780501 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771197    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 19:21:42.780501 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771199    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 19:21:42.780501 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771202    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 19:21:42.780501 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771204    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 19:21:42.780501 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771207    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 19:21:42.780501 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771211    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 19:21:42.780501 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771214    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 19:21:42.780501 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771217    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 19:21:42.780501 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771220    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 19:21:42.780501 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771223    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 19:21:42.780501 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771226    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 19:21:42.780501 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771228    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 19:21:42.780501 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771231    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 19:21:42.780985 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771233    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 19:21:42.780985 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771236    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 19:21:42.780985 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771238    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 19:21:42.780985 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771241    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 19:21:42.780985 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771243    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 19:21:42.780985 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771247    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 19:21:42.780985 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771250    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 19:21:42.780985 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771253    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 19:21:42.780985 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771256    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 19:21:42.780985 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771258    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 19:21:42.780985 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771260    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 19:21:42.780985 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771263    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 19:21:42.780985 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771265    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 19:21:42.780985 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771268    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 19:21:42.780985 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771271    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 19:21:42.780985 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771273    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 19:21:42.780985 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771276    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 19:21:42.780985 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771278    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 19:21:42.780985 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771281    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 19:21:42.781492 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771284    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 19:21:42.781492 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771286    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 19:21:42.781492 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771289    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 19:21:42.781492 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771291    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 19:21:42.781492 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771294    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 19:21:42.781492 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771296    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 19:21:42.781492 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771299    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 19:21:42.781492 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771301    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 19:21:42.781492 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771304    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 19:21:42.781492 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771309    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 19:21:42.781492 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771312    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 19:21:42.781492 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771315    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 19:21:42.781492 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771317    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 19:21:42.781492 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771320    2573 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 19:21:42.781492 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771324    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 19:21:42.781492 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771328    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 19:21:42.781492 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771331    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 19:21:42.781492 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771334    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 19:21:42.781492 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771338    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 19:21:42.781970 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771341    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 19:21:42.781970 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771344    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 19:21:42.781970 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771347    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 19:21:42.781970 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771350    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 19:21:42.781970 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.771353    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 19:21:42.781970 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.771359    2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 19:21:42.781970 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.777975    2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 13 19:21:42.781970 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.777992    2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 13 19:21:42.781970 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.778013    2573 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 13 19:21:42.781970 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.778023    2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 13 19:21:42.781970 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778079    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 19:21:42.781970 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778085    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 19:21:42.781970 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778088    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 19:21:42.781970 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778092    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 19:21:42.781970 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778095    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 19:21:42.782365 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778098    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 19:21:42.782365 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778102    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 19:21:42.782365 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778105    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 19:21:42.782365 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778107    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 19:21:42.782365 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778110    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 19:21:42.782365 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778113    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 19:21:42.782365 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778116    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 19:21:42.782365 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778118    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 19:21:42.782365 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778121    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 19:21:42.782365 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778124    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 19:21:42.782365 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778126    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 19:21:42.782365 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778129    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 19:21:42.782365 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778131    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 19:21:42.782365 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778134    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 19:21:42.782365 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778137    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 19:21:42.782365 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778139    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 19:21:42.782365 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778142    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 19:21:42.782365 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778145    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 19:21:42.782365 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778147    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 19:21:42.782365 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778150    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 19:21:42.782866 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778153    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 19:21:42.782866 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778155    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 19:21:42.782866 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778158    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 19:21:42.782866 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778161    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 19:21:42.782866 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778165    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 19:21:42.782866 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778168    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 19:21:42.782866 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778170    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 19:21:42.782866 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778173    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 19:21:42.782866 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778176    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 19:21:42.782866 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778178    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 19:21:42.782866 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778181    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 19:21:42.782866 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778184    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 19:21:42.782866 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778186    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 19:21:42.782866 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778189    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 19:21:42.782866 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778191    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 19:21:42.782866 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778194    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 19:21:42.782866 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778196    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 19:21:42.782866 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778199    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 19:21:42.782866 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778202    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 19:21:42.782866 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778204    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 19:21:42.783341 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778207    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 19:21:42.783341 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778209    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 19:21:42.783341 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778212    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 19:21:42.783341 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778215    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 19:21:42.783341 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778218    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 19:21:42.783341 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778221    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 19:21:42.783341 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778224    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 19:21:42.783341 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778228    2573 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 19:21:42.783341 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778231    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 19:21:42.783341 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778234    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 19:21:42.783341 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778236    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 19:21:42.783341 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778239    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 19:21:42.783341 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778241    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 19:21:42.783341 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778244    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 19:21:42.783341 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778247    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 19:21:42.783341 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778250    2573 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 19:21:42.783341 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778254    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 19:21:42.783341 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778257    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 19:21:42.783341 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778259    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 19:21:42.783341 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778262    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 19:21:42.783911 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778265    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 19:21:42.783911 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778267    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 19:21:42.783911 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778270    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 19:21:42.783911 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778272    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 19:21:42.783911 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778275    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 19:21:42.783911 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778277    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 19:21:42.783911 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778281    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 19:21:42.783911 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778285    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 19:21:42.783911 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778288    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 19:21:42.783911 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778291    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 19:21:42.783911 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778294    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 19:21:42.783911 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778297    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 19:21:42.783911 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778300    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 19:21:42.783911 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778303    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 19:21:42.783911 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778305    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 19:21:42.783911 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778308    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 19:21:42.783911 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778311    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 19:21:42.783911 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778314    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 19:21:42.783911 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778317    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 19:21:42.784368 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778319    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 19:21:42.784368 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778322    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 19:21:42.784368 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778325    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 19:21:42.784368 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778328    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 19:21:42.784368 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778332    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 19:21:42.784368 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778335    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 19:21:42.784368 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778338    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 19:21:42.784368 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778341    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 19:21:42.784368 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778344    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 19:21:42.784368 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778347    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 19:21:42.784368 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778350    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 19:21:42.784368 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778353    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 19:21:42.784368 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778356    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 19:21:42.784368 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778358    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 19:21:42.784368 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778361    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 19:21:42.784775 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.778366    2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 19:21:42.784775 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778500    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 19:21:42.784775 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778506    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 19:21:42.784775 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778509    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 19:21:42.784775 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778511    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 19:21:42.784775 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778515    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 19:21:42.784775 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778518    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 19:21:42.784775 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778522    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 19:21:42.784775 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778525    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 19:21:42.784775 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778528    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 19:21:42.784775 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778531    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 19:21:42.784775 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778533    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 19:21:42.784775 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778536    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 19:21:42.784775 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778539    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 19:21:42.785125 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778542    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 19:21:42.785125 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778544    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 19:21:42.785125 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778547    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 19:21:42.785125 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778550    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 19:21:42.785125 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778552    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 19:21:42.785125 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778555    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 19:21:42.785125 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778557    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 19:21:42.785125 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778560    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 19:21:42.785125 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778563    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 19:21:42.785125 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778565    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 19:21:42.785125 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778568    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 19:21:42.785125 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778571    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 19:21:42.785125 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778573    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 19:21:42.785125 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778576    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 19:21:42.785125 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778579    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 19:21:42.785125 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778582    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 19:21:42.785125 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778585    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 19:21:42.785125 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778587    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 19:21:42.785125 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778590    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 19:21:42.785125 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778592    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 19:21:42.785624 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778595    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 19:21:42.785624 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778597    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 19:21:42.785624 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778600    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 19:21:42.785624 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778604    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 19:21:42.785624 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778607    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 19:21:42.785624 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778610    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 19:21:42.785624 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778612    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 19:21:42.785624 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778616    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 19:21:42.785624 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778619    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 19:21:42.785624 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778622    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 19:21:42.785624 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778625    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 19:21:42.785624 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778627    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 19:21:42.785624 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778630    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 19:21:42.785624 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778649    2573 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 19:21:42.785624 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778653    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 19:21:42.785624 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778656    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 19:21:42.785624 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778659    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 19:21:42.785624 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778662    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 19:21:42.785624 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778665    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 19:21:42.786118 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778667    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 19:21:42.786118 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778670    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 19:21:42.786118 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778673    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 19:21:42.786118 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778676    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 19:21:42.786118 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778679    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 19:21:42.786118 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778682    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 19:21:42.786118 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778685    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 19:21:42.786118 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778688    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 19:21:42.786118 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778690    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 19:21:42.786118 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778694    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 19:21:42.786118 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778697    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 19:21:42.786118 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778699    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 19:21:42.786118 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778702    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 19:21:42.786118 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778705    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 19:21:42.786118 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778707    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 19:21:42.786118 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778709    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 19:21:42.786118 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778713    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 19:21:42.786118 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778715    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 19:21:42.786118 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778718    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 19:21:42.786118 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778720    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 19:21:42.786599 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778723    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 19:21:42.786599 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778726    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 19:21:42.786599 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778728    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 19:21:42.786599 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778731    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 19:21:42.786599 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778733    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 19:21:42.786599 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778736    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 19:21:42.786599 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778739    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 19:21:42.786599 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778742    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 19:21:42.786599 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778745    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 19:21:42.786599 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778747    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 19:21:42.786599 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778750    2573 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 19:21:42.786599 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778752    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 19:21:42.786599 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778755    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 19:21:42.786599 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778758    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 19:21:42.786599 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778760    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 19:21:42.786599 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778763    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 19:21:42.786599 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778766    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 19:21:42.786599 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778768    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 19:21:42.786599 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778771    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 19:21:42.786599 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778774    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 19:21:42.787430 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778777    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 19:21:42.787430 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778779    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 19:21:42.787430 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778782    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 19:21:42.787430 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778785    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 19:21:42.787430 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778788    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 19:21:42.787430 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778790    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 19:21:42.787430 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:42.778792    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 19:21:42.787430 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.778797    2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 19:21:42.787430 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.778958    2573 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 13 19:21:42.787430 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.780934    2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 13 19:21:42.787430 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.782181    2573 server.go:1019] "Starting client certificate rotation"
Jul 13 19:21:42.787430 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.782282    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 13 19:21:42.787430 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.782318    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 13 19:21:42.787807 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.784650    2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 13 19:21:42.789428 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.789404    2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 13 19:21:42.801748 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.801714    2573 log.go:25] "Validated CRI v1 runtime API"
Jul 13 19:21:42.804095 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.804068    2573 log.go:25] "Validated CRI v1 image API"
Jul 13 19:21:42.805959 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.805939    2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 13 19:21:42.808913 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.808889    2573 fs.go:135] Filesystem UUIDs: map[35f539c8-3058-4868-999d-e1149c299cd7:/dev/nvme0n1p3 6da2bf7f-92ac-4c2d-a87a-d802419a509d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 13 19:21:42.809009 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.808910    2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 13 19:21:42.814147 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.814023    2573 manager.go:216] Machine: {Timestamp:2026-07-13 19:21:42.812609201 +0000 UTC m=+0.326485798 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100353 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2605af42bf6871c363124422fa9b12 SystemUUID:ec2605af-42bf-6871-c363-124422fa9b12 BootID:2376c351-0418-4075-ba93-bd8b35bdb0cf Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:22:a9:5a:58:cf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:22:a9:5a:58:cf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:66:28:54:4f:42:4a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 13 19:21:42.814147 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.814134    2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 13 19:21:42.814308 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.814218    2573 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 13 19:21:42.817087 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.817057    2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 13 19:21:42.817269 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.817089    2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-91.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 13 19:21:42.817765 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.817752    2573 topology_manager.go:138] "Creating topology manager with none policy"
Jul 13 19:21:42.817817 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.817771    2573 container_manager_linux.go:309] "Creating device plugin manager"
Jul 13 19:21:42.817817 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.817782    2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 13 19:21:42.818498 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.818486    2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 13 19:21:42.819062 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.819045    2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 13 19:21:42.819791 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.819778    2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 13 19:21:42.822435 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.822420    2573 state_mem.go:36] "Initialized new in-memory state store"
Jul 13 19:21:42.822576 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.822565    2573 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 13 19:21:42.822654 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.822606    2573 kubelet.go:486] "Attempting to sync node with API server"
Jul 13 19:21:42.822654 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.822623    2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 13 19:21:42.822778 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.822662    2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 13 19:21:42.822778 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.822677    2573 kubelet.go:398] "Adding apiserver pod source"
Jul 13 19:21:42.822778 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.822690    2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 13 19:21:42.823140 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.823100    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 13 19:21:42.823140 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.823126    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 13 19:21:42.825352 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.825335    2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 13 19:21:42.827186 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.827172    2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 13 19:21:42.827226 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.827203    2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 13 19:21:42.827384 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.827373    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 13 19:21:42.827413 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.827389    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 13 19:21:42.827413 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.827396    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 13 19:21:42.827413 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.827402    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 13 19:21:42.827413 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.827411    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 13 19:21:42.827516 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.827418    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 13 19:21:42.827962 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.827953    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 13 19:21:42.827962 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.827962    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 13 19:21:42.828016 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.827969    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 13 19:21:42.828016 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.827975    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 13 19:21:42.828016 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.827984    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 13 19:21:42.828016 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.827992    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 13 19:21:42.828727 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.828715    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 13 19:21:42.828727 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.828727    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 13 19:21:42.832097 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.832080    2573 server.go:1268] "Started kubelet"
Jul 13 19:21:42.832246 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.832192    2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 13 19:21:42.832305 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.832269    2573 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 13 19:21:42.832582 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.832221    2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 13 19:21:42.833024 ip-10-0-143-91 systemd[1]: Started Kubernetes Kubelet.
Jul 13 19:21:42.833254 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.833236    2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 13 19:21:42.837222 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.837196    2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-91.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 13 19:21:42.837714 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:42.837684    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 13 19:21:42.837832 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:42.837779    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-91.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 13 19:21:42.838358 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:42.838334    2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 13 19:21:42.838731 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.838714    2573 server.go:310] "Adding debug handlers to kubelet server"
Jul 13 19:21:42.841755 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.841735    2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 13 19:21:42.841755 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.841753    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 13 19:21:42.841947 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:42.841848    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-91.ec2.internal\" not found"
Jul 13 19:21:42.842416 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:42.841426    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-91.ec2.internal.18c1efb86f146a87  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-91.ec2.internal,UID:ip-10-0-143-91.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-91.ec2.internal,},FirstTimestamp:2026-07-13 19:21:42.832056967 +0000 UTC m=+0.345933565,LastTimestamp:2026-07-13 19:21:42.832056967 +0000 UTC m=+0.345933565,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-91.ec2.internal,}"
Jul 13 19:21:42.842416 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.842400    2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 13 19:21:42.842548 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.842406    2573 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 13 19:21:42.842548 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.842436    2573 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 13 19:21:42.842653 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.842601    2573 reconstruct.go:97] "Volume reconstruction finished"
Jul 13 19:21:42.842653 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.842611    2573 reconciler.go:29] "Reconciler: start to sync state"
Jul 13 19:21:42.842653 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.842651    2573 factory.go:55] Registering systemd factory
Jul 13 19:21:42.842798 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.842666    2573 factory.go:223] Registration of the systemd container factory successfully
Jul 13 19:21:42.842918 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.842901    2573 factory.go:153] Registering CRI-O factory
Jul 13 19:21:42.842995 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.842921    2573 factory.go:223] Registration of the crio container factory successfully
Jul 13 19:21:42.842995 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.842991    2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 13 19:21:42.843090 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.843017    2573 factory.go:103] Registering Raw factory
Jul 13 19:21:42.843090 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.843031    2573 manager.go:1196] Started watching for new ooms in manager
Jul 13 19:21:42.843372 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.843361    2573 manager.go:349] Starting recovery of all containers
Jul 13 19:21:42.850744 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.850584    2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 13 19:21:42.852585 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.852545    2573 manager.go:354] Recovery completed
Jul 13 19:21:42.853601 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:42.853573    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 13 19:21:42.853850 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:42.853740    2573 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-91.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 13 19:21:42.857770 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.857753    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 19:21:42.859828 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.859751    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 19:21:42.859828 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.859785    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 19:21:42.859828 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.859804    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasSufficientPID"
Jul 13 19:21:42.860364 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.860346    2573 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 13 19:21:42.860364 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.860361    2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 13 19:21:42.860499 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.860380    2573 state_mem.go:36] "Initialized new in-memory state store"
Jul 13 19:21:42.860499 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.860386    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 19:21:42.861199 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.861184    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 19:21:42.861253 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.861211    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 19:21:42.861253 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.861220    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasSufficientPID"
Jul 13 19:21:42.861607 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.861594    2573 policy_none.go:49] "None policy: Start"
Jul 13 19:21:42.861667 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.861610    2573 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 13 19:21:42.861667 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.861621    2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 13 19:21:42.862656 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.862631    2573 policy_none.go:47] "Start"
Jul 13 19:21:42.869214 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:42.869097    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-91.ec2.internal.18c1efb870bb3dc9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-91.ec2.internal,UID:ip-10-0-143-91.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-91.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-91.ec2.internal,},FirstTimestamp:2026-07-13 19:21:42.859767241 +0000 UTC m=+0.373643844,LastTimestamp:2026-07-13 19:21:42.859767241 +0000 UTC m=+0.373643844,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-91.ec2.internal,}"
Jul 13 19:21:42.871365 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:42.871289    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-91.ec2.internal.18c1efb870bb96bf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-91.ec2.internal,UID:ip-10-0-143-91.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-91.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-91.ec2.internal,},FirstTimestamp:2026-07-13 19:21:42.859790015 +0000 UTC m=+0.373666612,LastTimestamp:2026-07-13 19:21:42.859790015 +0000 UTC m=+0.373666612,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-91.ec2.internal,}"
Jul 13 19:21:42.882559 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:42.882480    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-91.ec2.internal.18c1efb870bbdf00  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-91.ec2.internal,UID:ip-10-0-143-91.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-91.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-91.ec2.internal,},FirstTimestamp:2026-07-13 19:21:42.859808512 +0000 UTC m=+0.373685108,LastTimestamp:2026-07-13 19:21:42.859808512 +0000 UTC m=+0.373685108,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-91.ec2.internal,}"
Jul 13 19:21:42.884000 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.883852    2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ws7b9"
Jul 13 19:21:42.894525 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.894496    2573 manager.go:337] "Starting Device Plugin manager"
Jul 13 19:21:42.894525 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.894508    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ws7b9"
Jul 13 19:21:42.894718 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:42.894540    2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 13 19:21:42.894718 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.894553    2573 server.go:87] "Starting device plugin registration server"
Jul 13 19:21:42.894887 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.894873    2573 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 13 19:21:42.894952 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.894888    2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 13 19:21:42.895092 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.895073    2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 13 19:21:42.895194 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.895159    2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 13 19:21:42.895194 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.895174    2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 13 19:21:42.895703 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:42.895680    2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 13 19:21:42.895784 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:42.895739    2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-91.ec2.internal\" not found"
Jul 13 19:21:42.896953 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:42.896848    2573 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-91.ec2.internal.18c1efb870bb3dc9\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-91.ec2.internal.18c1efb870bb3dc9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-91.ec2.internal,UID:ip-10-0-143-91.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-91.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-91.ec2.internal,},FirstTimestamp:2026-07-13 19:21:42.859767241 +0000 UTC m=+0.373643844,LastTimestamp:2026-07-13 19:21:42.861198304 +0000 UTC m=+0.375074901,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-91.ec2.internal,}"
Jul 13 19:21:42.995791 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.995752    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 19:21:42.997256 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.996990    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 19:21:42.997256 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.997021    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 19:21:42.997256 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.997032    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasSufficientPID"
Jul 13 19:21:42.997256 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:42.997059    2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-91.ec2.internal"
Jul 13 19:21:43.006079 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.006052    2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-91.ec2.internal"
Jul 13 19:21:43.006159 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:43.006082    2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-91.ec2.internal\": node \"ip-10-0-143-91.ec2.internal\" not found"
Jul 13 19:21:43.041762 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.041682    2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 13 19:21:43.041762 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.041728    2573 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 13 19:21:43.041762 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.041750    2573 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 13 19:21:43.041929 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:43.041784    2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 13 19:21:43.043933 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.043909    2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 19:21:43.066905 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:43.066875    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-91.ec2.internal\" not found"
Jul 13 19:21:43.142580 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.142549    2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-91.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-91.ec2.internal"]
Jul 13 19:21:43.142715 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.142663    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 19:21:43.143782 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.143764    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 19:21:43.143872 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.143795    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 19:21:43.143872 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.143811    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasSufficientPID"
Jul 13 19:21:43.145040 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.145028    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 19:21:43.145173 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.145160    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-91.ec2.internal"
Jul 13 19:21:43.145211 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.145190    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 19:21:43.145761 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.145744    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 19:21:43.145860 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.145778    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 19:21:43.145860 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.145780    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 19:21:43.145860 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.145793    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasSufficientPID"
Jul 13 19:21:43.145860 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.145804    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 19:21:43.145860 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.145813    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasSufficientPID"
Jul 13 19:21:43.146894 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.146876    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-91.ec2.internal"
Jul 13 19:21:43.146973 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.146910    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 19:21:43.147596 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.147581    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 19:21:43.147682 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.147611    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 19:21:43.147682 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.147626    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasSufficientPID"
Jul 13 19:21:43.165253 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:43.165232    2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-91.ec2.internal\" not found" node="ip-10-0-143-91.ec2.internal"
Jul 13 19:21:43.167157 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:43.167140    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-91.ec2.internal\" not found"
Jul 13 19:21:43.169811 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:43.169795    2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-91.ec2.internal\" not found" node="ip-10-0-143-91.ec2.internal"
Jul 13 19:21:43.244691 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.244655    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e34e84288ef00fb262466bd5864c1618-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-91.ec2.internal\" (UID: \"e34e84288ef00fb262466bd5864c1618\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-91.ec2.internal"
Jul 13 19:21:43.244691 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.244694    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6db41158752585940a2ece7d4dc21de6-config\") pod \"kube-apiserver-proxy-ip-10-0-143-91.ec2.internal\" (UID: \"6db41158752585940a2ece7d4dc21de6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-91.ec2.internal"
Jul 13 19:21:43.244816 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.244717    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e34e84288ef00fb262466bd5864c1618-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-91.ec2.internal\" (UID: \"e34e84288ef00fb262466bd5864c1618\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-91.ec2.internal"
Jul 13 19:21:43.267742 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:43.267717    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-91.ec2.internal\" not found"
Jul 13 19:21:43.345058 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.344999    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e34e84288ef00fb262466bd5864c1618-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-91.ec2.internal\" (UID: \"e34e84288ef00fb262466bd5864c1618\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-91.ec2.internal"
Jul 13 19:21:43.345058 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.345007    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e34e84288ef00fb262466bd5864c1618-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-91.ec2.internal\" (UID: \"e34e84288ef00fb262466bd5864c1618\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-91.ec2.internal"
Jul 13 19:21:43.345150 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.345073    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e34e84288ef00fb262466bd5864c1618-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-91.ec2.internal\" (UID: \"e34e84288ef00fb262466bd5864c1618\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-91.ec2.internal"
Jul 13 19:21:43.345150 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.345092    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6db41158752585940a2ece7d4dc21de6-config\") pod \"kube-apiserver-proxy-ip-10-0-143-91.ec2.internal\" (UID: \"6db41158752585940a2ece7d4dc21de6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-91.ec2.internal"
Jul 13 19:21:43.345150 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.345120    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6db41158752585940a2ece7d4dc21de6-config\") pod \"kube-apiserver-proxy-ip-10-0-143-91.ec2.internal\" (UID: \"6db41158752585940a2ece7d4dc21de6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-91.ec2.internal"
Jul 13 19:21:43.345150 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.345136    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e34e84288ef00fb262466bd5864c1618-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-91.ec2.internal\" (UID: \"e34e84288ef00fb262466bd5864c1618\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-91.ec2.internal"
Jul 13 19:21:43.368099 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:43.368070    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-91.ec2.internal\" not found"
Jul 13 19:21:43.467699 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.467669    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-91.ec2.internal"
Jul 13 19:21:43.468728 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:43.468709    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-91.ec2.internal\" not found"
Jul 13 19:21:43.472783 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.472484    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-91.ec2.internal"
Jul 13 19:21:43.569752 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:43.569709    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-91.ec2.internal\" not found"
Jul 13 19:21:43.670298 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:43.670215    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-91.ec2.internal\" not found"
Jul 13 19:21:43.770808 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:43.770775    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-91.ec2.internal\" not found"
Jul 13 19:21:43.781954 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.781930    2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 13 19:21:43.782109 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.782092    2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 13 19:21:43.871545 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:43.871510    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-91.ec2.internal\" not found"
Jul 13 19:21:43.885711 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.885686    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 19:21:43.886800 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.886783    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 19:21:43.886912 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.886816    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 19:21:43.886912 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.886837    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeHasSufficientPID"
Jul 13 19:21:43.900029 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.899994    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 19:16:42 +0000 UTC" deadline="2028-03-09 19:15:55.671556263 +0000 UTC"
Jul 13 19:21:43.900029 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:43.900024    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14519h54m11.771535323s"
Jul 13 19:21:43.971973 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:43.971941    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-91.ec2.internal\" not found"
Jul 13 19:21:44.072314 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:44.072270    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-91.ec2.internal\" not found"
Jul 13 19:21:44.172758 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:44.172734    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-91.ec2.internal\" not found"
Jul 13 19:21:44.183953 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.183929    2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 19:21:44.273719 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:44.273690    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-91.ec2.internal\" not found"
Jul 13 19:21:44.302573 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.302548    2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 19:21:44.342929 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.342896    2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-91.ec2.internal"
Jul 13 19:21:44.360542 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:44.360509    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode34e84288ef00fb262466bd5864c1618.slice/crio-fe681339ed705200c784981cdb1696de65e7bc264138b5d6304ea7176ff7ee22 WatchSource:0}: Error finding container fe681339ed705200c784981cdb1696de65e7bc264138b5d6304ea7176ff7ee22: Status 404 returned error can't find the container with id fe681339ed705200c784981cdb1696de65e7bc264138b5d6304ea7176ff7ee22
Jul 13 19:21:44.360954 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:44.360934    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6db41158752585940a2ece7d4dc21de6.slice/crio-2f2cdb31e7f0b582a3af5407352fcc5f248b199e1bbe15236f7ffb028b12b631 WatchSource:0}: Error finding container 2f2cdb31e7f0b582a3af5407352fcc5f248b199e1bbe15236f7ffb028b12b631: Status 404 returned error can't find the container with id 2f2cdb31e7f0b582a3af5407352fcc5f248b199e1bbe15236f7ffb028b12b631
Jul 13 19:21:44.366309 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.366288    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 19:21:44.387240 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.385688    2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 13 19:21:44.387240 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.386964    2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-91.ec2.internal"
Jul 13 19:21:44.401297 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.401272    2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 13 19:21:44.451958 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.451930    2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 19:21:44.824220 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.824184    2573 apiserver.go:52] "Watching apiserver"
Jul 13 19:21:44.832624 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.832600    2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 13 19:21:44.832953 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.832932    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-91.ec2.internal","openshift-multus/multus-additional-cni-plugins-wsf2z","openshift-network-diagnostics/network-check-target-2q22b","openshift-network-operator/iptables-alerter-7gssd","kube-system/global-pull-secret-syncer-xxrt7","kube-system/konnectivity-agent-xdp7v","openshift-cluster-node-tuning-operator/tuned-zgxjr","openshift-dns/node-resolver-6kk2c","openshift-multus/multus-5dvcw","openshift-multus/network-metrics-daemon-hjx4z","openshift-ovn-kubernetes/ovnkube-node-cj8b5","kube-system/kube-apiserver-proxy-ip-10-0-143-91.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7","openshift-image-registry/node-ca-l6bw8"]
Jul 13 19:21:44.835307 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.835284    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.836360 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.836342    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:44.837714 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.837699    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:21:44.837782 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:44.837759    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2q22b" podUID="cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8"
Jul 13 19:21:44.838056 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.838039    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 13 19:21:44.838103 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.838058    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 13 19:21:44.838177 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.838162    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tgw8n\""
Jul 13 19:21:44.838615 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.838601    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-t9jw9\""
Jul 13 19:21:44.838769 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.838755    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 13 19:21:44.838838 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.838774    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 13 19:21:44.838893 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.838836    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 13 19:21:44.839032 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.839020    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7gssd"
Jul 13 19:21:44.840147 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.840129    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:21:44.840216 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:44.840190    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxrt7" podUID="690fc787-2061-47e9-8a05-7e596f4ddf0d"
Jul 13 19:21:44.841526 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.841480    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 13 19:21:44.841746 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.841646    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xdp7v"
Jul 13 19:21:44.841846 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.841761    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2q8pt\""
Jul 13 19:21:44.841846 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.841789    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 13 19:21:44.841992 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.841977    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 13 19:21:44.842147 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.842130    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 13 19:21:44.843335 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.843316    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.844620 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.844604    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-l6bw8"
Jul 13 19:21:44.845746 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.845730    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:44.846779 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.846731    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-q87nl\""
Jul 13 19:21:44.846877 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.846812    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 13 19:21:44.846877 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.846815    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 13 19:21:44.846877 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.846862    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zlklr\""
Jul 13 19:21:44.847032 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.846925    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 13 19:21:44.847032 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.846988    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 13 19:21:44.847180 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.847165    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 13 19:21:44.847180 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.847174    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 13 19:21:44.847279 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.847221    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 13 19:21:44.847279 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.847238    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 13 19:21:44.847772 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.847737    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:21:44.847864 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:44.847830    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjx4z" podUID="fff12175-e77b-4ec7-b0ca-eb4b45145f76"
Jul 13 19:21:44.847864 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.847854    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4gpmg\""
Jul 13 19:21:44.848037 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.848018    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 13 19:21:44.849659 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.849491    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 13 19:21:44.850142 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.849762    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 13 19:21:44.850142 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.849863    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:44.850405 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.850387    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4scwj\""
Jul 13 19:21:44.851392 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.850995    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qgqn\" (UniqueName: \"kubernetes.io/projected/bc244587-d876-4024-8165-a2727f5d0863-kube-api-access-6qgqn\") pod \"iptables-alerter-7gssd\" (UID: \"bc244587-d876-4024-8165-a2727f5d0863\") " pod="openshift-network-operator/iptables-alerter-7gssd"
Jul 13 19:21:44.851392 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.851044    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fad50d8c-d894-4a4d-a033-7ff1feb80dea-etc-selinux\") pod \"aws-ebs-csi-driver-node-kpqb7\" (UID: \"fad50d8c-d894-4a4d-a033-7ff1feb80dea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:44.851392 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.851076    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-host-run-multus-certs\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.851392 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.851110    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-multus-conf-dir\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.851392 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.851149    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6kk2c"
Jul 13 19:21:44.851392 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.851178    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-etc-systemd\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.851392 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.851211    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d389114a-4b4f-42f5-8d8c-8dc751a82553-etc-tuned\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.851392 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.851240    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fck2n\" (UniqueName: \"kubernetes.io/projected/d389114a-4b4f-42f5-8d8c-8dc751a82553-kube-api-access-fck2n\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.851392 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.851264    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-host-run-k8s-cni-cncf-io\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.851392 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.851312    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fad50d8c-d894-4a4d-a033-7ff1feb80dea-registration-dir\") pod \"aws-ebs-csi-driver-node-kpqb7\" (UID: \"fad50d8c-d894-4a4d-a033-7ff1feb80dea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:44.851392 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.851352    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9bjb\" (UniqueName: \"kubernetes.io/projected/fad50d8c-d894-4a4d-a033-7ff1feb80dea-kube-api-access-c9bjb\") pod \"aws-ebs-csi-driver-node-kpqb7\" (UID: \"fad50d8c-d894-4a4d-a033-7ff1feb80dea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:44.851956 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.851834    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/690fc787-2061-47e9-8a05-7e596f4ddf0d-kubelet-config\") pod \"global-pull-secret-syncer-xxrt7\" (UID: \"690fc787-2061-47e9-8a05-7e596f4ddf0d\") " pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:21:44.851956 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.851937    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-multus-cni-dir\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.852052 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.851980    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-etc-kubernetes\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.852052 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852005    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-etc-sysconfig\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.852052 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852048    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-etc-kubernetes\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.852190 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852071    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fad50d8c-d894-4a4d-a033-7ff1feb80dea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kpqb7\" (UID: \"fad50d8c-d894-4a4d-a033-7ff1feb80dea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:44.852424 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852404    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/15ae0b99-58a3-4072-beb3-abe52af62c10-konnectivity-ca\") pod \"konnectivity-agent-xdp7v\" (UID: \"15ae0b99-58a3-4072-beb3-abe52af62c10\") " pod="kube-system/konnectivity-agent-xdp7v"
Jul 13 19:21:44.852485 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852432    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-system-cni-dir\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.852485 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852449    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-multus-socket-dir-parent\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.852485 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852468    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-host-run-netns\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.852627 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852490    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-host-var-lib-cni-multus\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.852627 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852512    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-hostroot\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.852627 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852526    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmblc\" (UniqueName: \"kubernetes.io/projected/2971001d-8014-4210-aa64-f477c4062715-kube-api-access-cmblc\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.852627 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852540    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bc244587-d876-4024-8165-a2727f5d0863-iptables-alerter-script\") pod \"iptables-alerter-7gssd\" (UID: \"bc244587-d876-4024-8165-a2727f5d0863\") " pod="openshift-network-operator/iptables-alerter-7gssd"
Jul 13 19:21:44.852627 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852558    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlxgj\" (UniqueName: \"kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj\") pod \"network-check-target-2q22b\" (UID: \"cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8\") " pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:21:44.852627 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852581    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/690fc787-2061-47e9-8a05-7e596f4ddf0d-dbus\") pod \"global-pull-secret-syncer-xxrt7\" (UID: \"690fc787-2061-47e9-8a05-7e596f4ddf0d\") " pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:21:44.852627 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852614    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-host-var-lib-kubelet\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.852942 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852662    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-run\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.852942 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852681    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fad50d8c-d894-4a4d-a033-7ff1feb80dea-device-dir\") pod \"aws-ebs-csi-driver-node-kpqb7\" (UID: \"fad50d8c-d894-4a4d-a033-7ff1feb80dea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:44.852942 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852707    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/15ae0b99-58a3-4072-beb3-abe52af62c10-agent-certs\") pod \"konnectivity-agent-xdp7v\" (UID: \"15ae0b99-58a3-4072-beb3-abe52af62c10\") " pod="kube-system/konnectivity-agent-xdp7v"
Jul 13 19:21:44.852942 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852729    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-os-release\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.852942 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852751    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-etc-modprobe-d\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.852942 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852772    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-etc-sysctl-d\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.852942 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852793    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-etc-sysctl-conf\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.852942 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852811    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-var-lib-kubelet\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.852942 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852832    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-host\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.852942 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852848    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc244587-d876-4024-8165-a2727f5d0863-host-slash\") pod \"iptables-alerter-7gssd\" (UID: \"bc244587-d876-4024-8165-a2727f5d0863\") " pod="openshift-network-operator/iptables-alerter-7gssd"
Jul 13 19:21:44.852942 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852864    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fad50d8c-d894-4a4d-a033-7ff1feb80dea-socket-dir\") pod \"aws-ebs-csi-driver-node-kpqb7\" (UID: \"fad50d8c-d894-4a4d-a033-7ff1feb80dea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:44.852942 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852878    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fad50d8c-d894-4a4d-a033-7ff1feb80dea-sys-fs\") pod \"aws-ebs-csi-driver-node-kpqb7\" (UID: \"fad50d8c-d894-4a4d-a033-7ff1feb80dea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:44.852942 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852897    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-sys\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.853505 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852947    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-lib-modules\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.853505 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.852977    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d389114a-4b4f-42f5-8d8c-8dc751a82553-tmp\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.853505 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.853005    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret\") pod \"global-pull-secret-syncer-xxrt7\" (UID: \"690fc787-2061-47e9-8a05-7e596f4ddf0d\") " pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:21:44.853505 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.853021    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-cnibin\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.853505 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.853036    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2971001d-8014-4210-aa64-f477c4062715-cni-binary-copy\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.853505 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.853054    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-host-var-lib-cni-bin\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.853505 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.853072    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2971001d-8014-4210-aa64-f477c4062715-multus-daemon-config\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.853505 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.853312    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 13 19:21:44.853938 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.853555    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 13 19:21:44.853938 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.853565    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 13 19:21:44.853938 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.853577    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 13 19:21:44.853938 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.853583    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7d5b6\""
Jul 13 19:21:44.853938 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.853906    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 13 19:21:44.853938 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.853915    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 13 19:21:44.854207 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.853993    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f8cz8\""
Jul 13 19:21:44.854207 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.854017    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 13 19:21:44.854308 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.854269    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 13 19:21:44.861373 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.861354    2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 13 19:21:44.895347 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.895327    2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-w59g8"
Jul 13 19:21:44.906576 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.906556    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-w59g8"
Jul 13 19:21:44.943079 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.943048    2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 13 19:21:44.953363 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953339    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlxgj\" (UniqueName: \"kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj\") pod \"network-check-target-2q22b\" (UID: \"cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8\") " pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:21:44.953427 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953370    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/690fc787-2061-47e9-8a05-7e596f4ddf0d-dbus\") pod \"global-pull-secret-syncer-xxrt7\" (UID: \"690fc787-2061-47e9-8a05-7e596f4ddf0d\") " pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:21:44.953427 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953394    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-host-var-lib-kubelet\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.953427 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953415    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxhx9\" (UniqueName: \"kubernetes.io/projected/fff12175-e77b-4ec7-b0ca-eb4b45145f76-kube-api-access-zxhx9\") pod \"network-metrics-daemon-hjx4z\" (UID: \"fff12175-e77b-4ec7-b0ca-eb4b45145f76\") " pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:21:44.953625 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953432    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-node-log\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:44.953625 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953447    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-host-run-ovn-kubernetes\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:44.953625 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953468    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3d0d821f-0a58-4566-af42-eca2019fa233-ovn-node-metrics-cert\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:44.953625 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953484    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-host-var-lib-kubelet\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.953625 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953489    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fad50d8c-d894-4a4d-a033-7ff1feb80dea-device-dir\") pod \"aws-ebs-csi-driver-node-kpqb7\" (UID: \"fad50d8c-d894-4a4d-a033-7ff1feb80dea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:44.953625 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953532    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-os-release\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.953625 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953539    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fad50d8c-d894-4a4d-a033-7ff1feb80dea-device-dir\") pod \"aws-ebs-csi-driver-node-kpqb7\" (UID: \"fad50d8c-d894-4a4d-a033-7ff1feb80dea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:44.953625 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953557    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/690fc787-2061-47e9-8a05-7e596f4ddf0d-dbus\") pod \"global-pull-secret-syncer-xxrt7\" (UID: \"690fc787-2061-47e9-8a05-7e596f4ddf0d\") " pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:21:44.953625 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953564    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-etc-modprobe-d\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.953625 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953595    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-os-release\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.953625 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953598    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d389114a-4b4f-42f5-8d8c-8dc751a82553-tmp\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.953625 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953620    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-host-kubelet\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:44.954111 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953660    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3d0d821f-0a58-4566-af42-eca2019fa233-ovnkube-config\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:44.954111 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953683    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d300bbe-5196-46b2-96a3-a35f690f0207-tmp-dir\") pod \"node-resolver-6kk2c\" (UID: \"9d300bbe-5196-46b2-96a3-a35f690f0207\") " pod="openshift-dns/node-resolver-6kk2c"
Jul 13 19:21:44.954111 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953700    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-lib-modules\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.954111 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953684    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-etc-modprobe-d\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.954111 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953726    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc8xp\" (UniqueName: \"kubernetes.io/projected/86a1f531-5f82-4b77-b776-762ae3ff6170-kube-api-access-jc8xp\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:44.954111 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953826    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-host-slash\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:44.954111 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953863    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-host-cni-bin\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:44.954111 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953833    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-lib-modules\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.954111 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953891    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:44.954111 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953912    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret\") pod \"global-pull-secret-syncer-xxrt7\" (UID: \"690fc787-2061-47e9-8a05-7e596f4ddf0d\") " pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:21:44.954111 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953929    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2971001d-8014-4210-aa64-f477c4062715-cni-binary-copy\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.954111 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953955    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2971001d-8014-4210-aa64-f477c4062715-multus-daemon-config\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.954111 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953984    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fad50d8c-d894-4a4d-a033-7ff1feb80dea-etc-selinux\") pod \"aws-ebs-csi-driver-node-kpqb7\" (UID: \"fad50d8c-d894-4a4d-a033-7ff1feb80dea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:44.954111 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954009    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-systemd-units\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:44.954111 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954035    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-host-cni-netd\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:44.954111 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.953956    2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 13 19:21:44.954111 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954057    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pqgj\" (UniqueName: \"kubernetes.io/projected/9d300bbe-5196-46b2-96a3-a35f690f0207-kube-api-access-8pqgj\") pod \"node-resolver-6kk2c\" (UID: \"9d300bbe-5196-46b2-96a3-a35f690f0207\") " pod="openshift-dns/node-resolver-6kk2c"
Jul 13 19:21:44.954111 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:44.954106    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 19:21:44.954847 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954135    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-multus-conf-dir\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.954847 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:44.954177    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret podName:690fc787-2061-47e9-8a05-7e596f4ddf0d nodeName:}" failed. No retries permitted until 2026-07-13 19:21:45.454151841 +0000 UTC m=+2.968028457 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret") pod "global-pull-secret-syncer-xxrt7" (UID: "690fc787-2061-47e9-8a05-7e596f4ddf0d") : object "kube-system"/"original-pull-secret" not registered
Jul 13 19:21:44.954847 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954110    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fad50d8c-d894-4a4d-a033-7ff1feb80dea-etc-selinux\") pod \"aws-ebs-csi-driver-node-kpqb7\" (UID: \"fad50d8c-d894-4a4d-a033-7ff1feb80dea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:44.954847 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954198    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fck2n\" (UniqueName: \"kubernetes.io/projected/d389114a-4b4f-42f5-8d8c-8dc751a82553-kube-api-access-fck2n\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.954847 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954235    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-var-lib-kubelet\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.954847 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954179    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-multus-conf-dir\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.954847 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954261    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11a0b6c6-aae7-459a-9463-95459fc8ab61-host\") pod \"node-ca-l6bw8\" (UID: \"11a0b6c6-aae7-459a-9463-95459fc8ab61\") " pod="openshift-image-registry/node-ca-l6bw8"
Jul 13 19:21:44.954847 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954304    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fad50d8c-d894-4a4d-a033-7ff1feb80dea-registration-dir\") pod \"aws-ebs-csi-driver-node-kpqb7\" (UID: \"fad50d8c-d894-4a4d-a033-7ff1feb80dea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:44.954847 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954320    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-var-lib-kubelet\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.954847 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954334    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9bjb\" (UniqueName: \"kubernetes.io/projected/fad50d8c-d894-4a4d-a033-7ff1feb80dea-kube-api-access-c9bjb\") pod \"aws-ebs-csi-driver-node-kpqb7\" (UID: \"fad50d8c-d894-4a4d-a033-7ff1feb80dea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:44.954847 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954393    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/690fc787-2061-47e9-8a05-7e596f4ddf0d-kubelet-config\") pod \"global-pull-secret-syncer-xxrt7\" (UID: \"690fc787-2061-47e9-8a05-7e596f4ddf0d\") " pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:21:44.954847 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954408    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fad50d8c-d894-4a4d-a033-7ff1feb80dea-registration-dir\") pod \"aws-ebs-csi-driver-node-kpqb7\" (UID: \"fad50d8c-d894-4a4d-a033-7ff1feb80dea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:44.954847 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954419    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-multus-cni-dir\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.954847 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954450    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-etc-kubernetes\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.954847 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954460    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-multus-cni-dir\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.954847 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954477    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-run-ovn\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:44.955508 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954493    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2971001d-8014-4210-aa64-f477c4062715-multus-daemon-config\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.955508 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954505    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-etc-kubernetes\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.955508 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954507    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/690fc787-2061-47e9-8a05-7e596f4ddf0d-kubelet-config\") pod \"global-pull-secret-syncer-xxrt7\" (UID: \"690fc787-2061-47e9-8a05-7e596f4ddf0d\") " pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:21:44.955508 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954505    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86a1f531-5f82-4b77-b776-762ae3ff6170-system-cni-dir\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:44.955508 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954561    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fad50d8c-d894-4a4d-a033-7ff1feb80dea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kpqb7\" (UID: \"fad50d8c-d894-4a4d-a033-7ff1feb80dea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:44.955508 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954608    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/15ae0b99-58a3-4072-beb3-abe52af62c10-konnectivity-ca\") pod \"konnectivity-agent-xdp7v\" (UID: \"15ae0b99-58a3-4072-beb3-abe52af62c10\") " pod="kube-system/konnectivity-agent-xdp7v"
Jul 13 19:21:44.955508 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954606    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fad50d8c-d894-4a4d-a033-7ff1feb80dea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kpqb7\" (UID: \"fad50d8c-d894-4a4d-a033-7ff1feb80dea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:44.955508 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954716    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-host-run-netns\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.955508 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954751    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-host-var-lib-cni-multus\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.955508 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954768    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-run-systemd\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:44.955508 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954785    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqc4v\" (UniqueName: \"kubernetes.io/projected/3d0d821f-0a58-4566-af42-eca2019fa233-kube-api-access-qqc4v\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:44.955508 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954820    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bc244587-d876-4024-8165-a2727f5d0863-iptables-alerter-script\") pod \"iptables-alerter-7gssd\" (UID: \"bc244587-d876-4024-8165-a2727f5d0863\") " pod="openshift-network-operator/iptables-alerter-7gssd"
Jul 13 19:21:44.955508 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954831    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-host-var-lib-cni-multus\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.955508 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954846    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-host-run-netns\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.955508 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954879    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs\") pod \"network-metrics-daemon-hjx4z\" (UID: \"fff12175-e77b-4ec7-b0ca-eb4b45145f76\") " pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:21:44.955508 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954473    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2971001d-8014-4210-aa64-f477c4062715-cni-binary-copy\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.955508 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954930    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/86a1f531-5f82-4b77-b776-762ae3ff6170-cni-binary-copy\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:44.956242 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.954959    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3d0d821f-0a58-4566-af42-eca2019fa233-ovnkube-script-lib\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:44.956242 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955007    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/15ae0b99-58a3-4072-beb3-abe52af62c10-agent-certs\") pod \"konnectivity-agent-xdp7v\" (UID: \"15ae0b99-58a3-4072-beb3-abe52af62c10\") " pod="kube-system/konnectivity-agent-xdp7v"
Jul 13 19:21:44.956242 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955033    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-etc-sysctl-conf\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.956242 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955077    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9d300bbe-5196-46b2-96a3-a35f690f0207-hosts-file\") pod \"node-resolver-6kk2c\" (UID: \"9d300bbe-5196-46b2-96a3-a35f690f0207\") " pod="openshift-dns/node-resolver-6kk2c"
Jul 13 19:21:44.956242 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955084    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/15ae0b99-58a3-4072-beb3-abe52af62c10-konnectivity-ca\") pod \"konnectivity-agent-xdp7v\" (UID: \"15ae0b99-58a3-4072-beb3-abe52af62c10\") " pod="kube-system/konnectivity-agent-xdp7v"
Jul 13 19:21:44.956242 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955142    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc244587-d876-4024-8165-a2727f5d0863-host-slash\") pod \"iptables-alerter-7gssd\" (UID: \"bc244587-d876-4024-8165-a2727f5d0863\") " pod="openshift-network-operator/iptables-alerter-7gssd"
Jul 13 19:21:44.956242 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955183    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-etc-sysctl-conf\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.956242 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955214    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fad50d8c-d894-4a4d-a033-7ff1feb80dea-socket-dir\") pod \"aws-ebs-csi-driver-node-kpqb7\" (UID: \"fad50d8c-d894-4a4d-a033-7ff1feb80dea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:44.956242 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955243    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc244587-d876-4024-8165-a2727f5d0863-host-slash\") pod \"iptables-alerter-7gssd\" (UID: \"bc244587-d876-4024-8165-a2727f5d0863\") " pod="openshift-network-operator/iptables-alerter-7gssd"
Jul 13 19:21:44.956242 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955262    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fad50d8c-d894-4a4d-a033-7ff1feb80dea-sys-fs\") pod \"aws-ebs-csi-driver-node-kpqb7\" (UID: \"fad50d8c-d894-4a4d-a033-7ff1feb80dea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:44.956242 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955312    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-sys\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.956242 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955341    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fad50d8c-d894-4a4d-a033-7ff1feb80dea-socket-dir\") pod \"aws-ebs-csi-driver-node-kpqb7\" (UID: \"fad50d8c-d894-4a4d-a033-7ff1feb80dea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:44.956242 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955348    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bc244587-d876-4024-8165-a2727f5d0863-iptables-alerter-script\") pod \"iptables-alerter-7gssd\" (UID: \"bc244587-d876-4024-8165-a2727f5d0863\") " pod="openshift-network-operator/iptables-alerter-7gssd"
Jul 13 19:21:44.956242 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955346    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fad50d8c-d894-4a4d-a033-7ff1feb80dea-sys-fs\") pod \"aws-ebs-csi-driver-node-kpqb7\" (UID: \"fad50d8c-d894-4a4d-a033-7ff1feb80dea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:44.956242 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955385    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/86a1f531-5f82-4b77-b776-762ae3ff6170-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:44.956242 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955410    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-sys\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.956242 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955452    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-cnibin\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.956785 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955489    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-cnibin\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.956785 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955491    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-host-var-lib-cni-bin\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.956785 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955521    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-host-run-netns\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:44.956785 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955527    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-host-var-lib-cni-bin\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.956785 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955542    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-log-socket\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:44.956785 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955566    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3d0d821f-0a58-4566-af42-eca2019fa233-env-overrides\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:44.956785 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955585    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qgqn\" (UniqueName: \"kubernetes.io/projected/bc244587-d876-4024-8165-a2727f5d0863-kube-api-access-6qgqn\") pod \"iptables-alerter-7gssd\" (UID: \"bc244587-d876-4024-8165-a2727f5d0863\") " pod="openshift-network-operator/iptables-alerter-7gssd"
Jul 13 19:21:44.956785 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955604    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-host-run-multus-certs\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.956785 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955647    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/86a1f531-5f82-4b77-b776-762ae3ff6170-os-release\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:44.956785 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955663    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-host-run-multus-certs\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.956785 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955673    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-etc-systemd\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.956785 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955697    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d389114a-4b4f-42f5-8d8c-8dc751a82553-etc-tuned\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.956785 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955713    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/11a0b6c6-aae7-459a-9463-95459fc8ab61-serviceca\") pod \"node-ca-l6bw8\" (UID: \"11a0b6c6-aae7-459a-9463-95459fc8ab61\") " pod="openshift-image-registry/node-ca-l6bw8"
Jul 13 19:21:44.956785 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955728    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-var-lib-openvswitch\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:44.956785 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955744    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-etc-sysctl-d\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.956785 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955766    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-host\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.956785 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955789    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6rgv\" (UniqueName: \"kubernetes.io/projected/11a0b6c6-aae7-459a-9463-95459fc8ab61-kube-api-access-p6rgv\") pod \"node-ca-l6bw8\" (UID: \"11a0b6c6-aae7-459a-9463-95459fc8ab61\") " pod="openshift-image-registry/node-ca-l6bw8"
Jul 13 19:21:44.957264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955813    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-host-run-k8s-cni-cncf-io\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.957264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955836    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-etc-sysconfig\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.957264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955837    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-host\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.957264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955766    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-etc-systemd\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.957264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955855    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-etc-sysctl-d\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.957264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955875    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-etc-kubernetes\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.957264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955900    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-etc-sysconfig\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.957264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955903    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-host-run-k8s-cni-cncf-io\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.957264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955905    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/86a1f531-5f82-4b77-b776-762ae3ff6170-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:44.957264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955931    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-etc-kubernetes\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.957264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955945    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/86a1f531-5f82-4b77-b776-762ae3ff6170-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:44.957264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.955979    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-system-cni-dir\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.957264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.956012    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-multus-socket-dir-parent\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.957264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.956035    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-hostroot\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.957264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.956040    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-system-cni-dir\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.957264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.956057    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmblc\" (UniqueName: \"kubernetes.io/projected/2971001d-8014-4210-aa64-f477c4062715-kube-api-access-cmblc\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.957264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.956083    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-run\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.957264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.956086    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-multus-socket-dir-parent\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.957803 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.956108    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/86a1f531-5f82-4b77-b776-762ae3ff6170-cnibin\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:44.957803 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.956138    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-etc-openvswitch\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:44.957803 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.956162    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-run-openvswitch\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:44.957803 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.956166    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d389114a-4b4f-42f5-8d8c-8dc751a82553-run\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.957803 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.956309    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2971001d-8014-4210-aa64-f477c4062715-hostroot\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.957803 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.957392    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d389114a-4b4f-42f5-8d8c-8dc751a82553-tmp\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.957803 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.957782    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/15ae0b99-58a3-4072-beb3-abe52af62c10-agent-certs\") pod \"konnectivity-agent-xdp7v\" (UID: \"15ae0b99-58a3-4072-beb3-abe52af62c10\") " pod="kube-system/konnectivity-agent-xdp7v"
Jul 13 19:21:44.958023 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.957891    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d389114a-4b4f-42f5-8d8c-8dc751a82553-etc-tuned\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:44.965223 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:44.965203    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 19:21:44.965223 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:44.965226    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 19:21:44.965355 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:44.965236    2573 projected.go:196] Error preparing data for projected volume kube-api-access-dlxgj for pod openshift-network-diagnostics/network-check-target-2q22b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:21:44.965355 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:44.965294    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj podName:cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8 nodeName:}" failed. No retries permitted until 2026-07-13 19:21:45.465277973 +0000 UTC m=+2.979154571 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dlxgj" (UniqueName: "kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj") pod "network-check-target-2q22b" (UID: "cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:21:44.966706 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.966684    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9bjb\" (UniqueName: \"kubernetes.io/projected/fad50d8c-d894-4a4d-a033-7ff1feb80dea-kube-api-access-c9bjb\") pod \"aws-ebs-csi-driver-node-kpqb7\" (UID: \"fad50d8c-d894-4a4d-a033-7ff1feb80dea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:44.969106 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.969080    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmblc\" (UniqueName: \"kubernetes.io/projected/2971001d-8014-4210-aa64-f477c4062715-kube-api-access-cmblc\") pod \"multus-5dvcw\" (UID: \"2971001d-8014-4210-aa64-f477c4062715\") " pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:44.969280 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.969255    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qgqn\" (UniqueName: \"kubernetes.io/projected/bc244587-d876-4024-8165-a2727f5d0863-kube-api-access-6qgqn\") pod \"iptables-alerter-7gssd\" (UID: \"bc244587-d876-4024-8165-a2727f5d0863\") " pod="openshift-network-operator/iptables-alerter-7gssd"
Jul 13 19:21:44.969623 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:44.969591    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fck2n\" (UniqueName: \"kubernetes.io/projected/d389114a-4b4f-42f5-8d8c-8dc751a82553-kube-api-access-fck2n\") pod \"tuned-zgxjr\" (UID: \"d389114a-4b4f-42f5-8d8c-8dc751a82553\") " pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:45.045783 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.045727    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-91.ec2.internal" event={"ID":"e34e84288ef00fb262466bd5864c1618","Type":"ContainerStarted","Data":"fe681339ed705200c784981cdb1696de65e7bc264138b5d6304ea7176ff7ee22"}
Jul 13 19:21:45.047195 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.047170    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-91.ec2.internal" event={"ID":"6db41158752585940a2ece7d4dc21de6","Type":"ContainerStarted","Data":"2f2cdb31e7f0b582a3af5407352fcc5f248b199e1bbe15236f7ffb028b12b631"}
Jul 13 19:21:45.056796 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.056761    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11a0b6c6-aae7-459a-9463-95459fc8ab61-host\") pod \"node-ca-l6bw8\" (UID: \"11a0b6c6-aae7-459a-9463-95459fc8ab61\") " pod="openshift-image-registry/node-ca-l6bw8"
Jul 13 19:21:45.056960 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.056811    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-run-ovn\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.056960 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.056837    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86a1f531-5f82-4b77-b776-762ae3ff6170-system-cni-dir\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:45.056960 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.056865    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-run-systemd\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.056960 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.056891    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqc4v\" (UniqueName: \"kubernetes.io/projected/3d0d821f-0a58-4566-af42-eca2019fa233-kube-api-access-qqc4v\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.056960 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.056900    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-run-ovn\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.056960 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.056916    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs\") pod \"network-metrics-daemon-hjx4z\" (UID: \"fff12175-e77b-4ec7-b0ca-eb4b45145f76\") " pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:21:45.056960 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.056916    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-run-systemd\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.056960 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.056941    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/86a1f531-5f82-4b77-b776-762ae3ff6170-cni-binary-copy\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:45.057347 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.056962    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86a1f531-5f82-4b77-b776-762ae3ff6170-system-cni-dir\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:45.057347 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.056965    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3d0d821f-0a58-4566-af42-eca2019fa233-ovnkube-script-lib\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.057347 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.056910    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11a0b6c6-aae7-459a-9463-95459fc8ab61-host\") pod \"node-ca-l6bw8\" (UID: \"11a0b6c6-aae7-459a-9463-95459fc8ab61\") " pod="openshift-image-registry/node-ca-l6bw8"
Jul 13 19:21:45.057347 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057009    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9d300bbe-5196-46b2-96a3-a35f690f0207-hosts-file\") pod \"node-resolver-6kk2c\" (UID: \"9d300bbe-5196-46b2-96a3-a35f690f0207\") " pod="openshift-dns/node-resolver-6kk2c"
Jul 13 19:21:45.057347 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057042    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/86a1f531-5f82-4b77-b776-762ae3ff6170-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:45.057347 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057069    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-host-run-netns\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.057347 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057094    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-log-socket\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.057347 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057114    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3d0d821f-0a58-4566-af42-eca2019fa233-env-overrides\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.057347 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057129    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9d300bbe-5196-46b2-96a3-a35f690f0207-hosts-file\") pod \"node-resolver-6kk2c\" (UID: \"9d300bbe-5196-46b2-96a3-a35f690f0207\") " pod="openshift-dns/node-resolver-6kk2c"
Jul 13 19:21:45.057347 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057137    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/86a1f531-5f82-4b77-b776-762ae3ff6170-os-release\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:45.057347 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:45.057013    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:21:45.057347 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057199    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/86a1f531-5f82-4b77-b776-762ae3ff6170-os-release\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:45.057347 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057238    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-log-socket\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.057347 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:45.057243    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs podName:fff12175-e77b-4ec7-b0ca-eb4b45145f76 nodeName:}" failed. No retries permitted until 2026-07-13 19:21:45.557221938 +0000 UTC m=+3.071098525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs") pod "network-metrics-daemon-hjx4z" (UID: "fff12175-e77b-4ec7-b0ca-eb4b45145f76") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:21:45.058009 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057410    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-host-run-netns\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.058009 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057178    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/11a0b6c6-aae7-459a-9463-95459fc8ab61-serviceca\") pod \"node-ca-l6bw8\" (UID: \"11a0b6c6-aae7-459a-9463-95459fc8ab61\") " pod="openshift-image-registry/node-ca-l6bw8"
Jul 13 19:21:45.058009 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057444    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-var-lib-openvswitch\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.058009 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057463    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6rgv\" (UniqueName: \"kubernetes.io/projected/11a0b6c6-aae7-459a-9463-95459fc8ab61-kube-api-access-p6rgv\") pod \"node-ca-l6bw8\" (UID: \"11a0b6c6-aae7-459a-9463-95459fc8ab61\") " pod="openshift-image-registry/node-ca-l6bw8"
Jul 13 19:21:45.058009 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057480    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/86a1f531-5f82-4b77-b776-762ae3ff6170-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:45.058009 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057496    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/86a1f531-5f82-4b77-b776-762ae3ff6170-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:45.058009 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057519    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/86a1f531-5f82-4b77-b776-762ae3ff6170-cnibin\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:45.058009 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057539    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-etc-openvswitch\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.058009 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057553    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-run-openvswitch\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.058009 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057581    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxhx9\" (UniqueName: \"kubernetes.io/projected/fff12175-e77b-4ec7-b0ca-eb4b45145f76-kube-api-access-zxhx9\") pod \"network-metrics-daemon-hjx4z\" (UID: \"fff12175-e77b-4ec7-b0ca-eb4b45145f76\") " pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:21:45.058009 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057596    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-node-log\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.058009 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057613    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-host-run-ovn-kubernetes\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.058009 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057613    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3d0d821f-0a58-4566-af42-eca2019fa233-env-overrides\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.058009 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057627    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3d0d821f-0a58-4566-af42-eca2019fa233-ovn-node-metrics-cert\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.058009 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057658    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/11a0b6c6-aae7-459a-9463-95459fc8ab61-serviceca\") pod \"node-ca-l6bw8\" (UID: \"11a0b6c6-aae7-459a-9463-95459fc8ab61\") " pod="openshift-image-registry/node-ca-l6bw8"
Jul 13 19:21:45.058009 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057667    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-host-kubelet\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.058009 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057683    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3d0d821f-0a58-4566-af42-eca2019fa233-ovnkube-config\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.058690 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057698    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d300bbe-5196-46b2-96a3-a35f690f0207-tmp-dir\") pod \"node-resolver-6kk2c\" (UID: \"9d300bbe-5196-46b2-96a3-a35f690f0207\") " pod="openshift-dns/node-resolver-6kk2c"
Jul 13 19:21:45.058690 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057715    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc8xp\" (UniqueName: \"kubernetes.io/projected/86a1f531-5f82-4b77-b776-762ae3ff6170-kube-api-access-jc8xp\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:45.058690 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057710    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/86a1f531-5f82-4b77-b776-762ae3ff6170-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:45.058690 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057755    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3d0d821f-0a58-4566-af42-eca2019fa233-ovnkube-script-lib\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.058690 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057768    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/86a1f531-5f82-4b77-b776-762ae3ff6170-cnibin\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:45.058690 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057772    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/86a1f531-5f82-4b77-b776-762ae3ff6170-cni-binary-copy\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:45.058690 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057795    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-host-kubelet\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.058690 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057696    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-var-lib-openvswitch\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.058690 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.057826    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/86a1f531-5f82-4b77-b776-762ae3ff6170-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:45.058690 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.058019    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-etc-openvswitch\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.058690 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.058058    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-run-openvswitch\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.058690 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.058054    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d300bbe-5196-46b2-96a3-a35f690f0207-tmp-dir\") pod \"node-resolver-6kk2c\" (UID: \"9d300bbe-5196-46b2-96a3-a35f690f0207\") " pod="openshift-dns/node-resolver-6kk2c"
Jul 13 19:21:45.058690 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.058095    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-node-log\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.058690 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.058113    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-host-run-ovn-kubernetes\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.058690 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.058138    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/86a1f531-5f82-4b77-b776-762ae3ff6170-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:45.058690 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.058170    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-host-slash\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.058690 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.058206    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-host-cni-bin\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.059356 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.058212    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-host-slash\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.059356 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.058231    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.059356 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.058260    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-host-cni-bin\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.059356 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.058270    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-systemd-units\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.059356 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.058286    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3d0d821f-0a58-4566-af42-eca2019fa233-ovnkube-config\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.059356 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.058278    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.059356 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.058291    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-host-cni-netd\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.059356 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.058328    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-systemd-units\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.059356 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.058368    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3d0d821f-0a58-4566-af42-eca2019fa233-host-cni-netd\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.059356 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.058389    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pqgj\" (UniqueName: \"kubernetes.io/projected/9d300bbe-5196-46b2-96a3-a35f690f0207-kube-api-access-8pqgj\") pod \"node-resolver-6kk2c\" (UID: \"9d300bbe-5196-46b2-96a3-a35f690f0207\") " pod="openshift-dns/node-resolver-6kk2c"
Jul 13 19:21:45.060163 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.060142    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3d0d821f-0a58-4566-af42-eca2019fa233-ovn-node-metrics-cert\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.070137 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.070111    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqc4v\" (UniqueName: \"kubernetes.io/projected/3d0d821f-0a58-4566-af42-eca2019fa233-kube-api-access-qqc4v\") pod \"ovnkube-node-cj8b5\" (UID: \"3d0d821f-0a58-4566-af42-eca2019fa233\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.070776 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.070748    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc8xp\" (UniqueName: \"kubernetes.io/projected/86a1f531-5f82-4b77-b776-762ae3ff6170-kube-api-access-jc8xp\") pod \"multus-additional-cni-plugins-wsf2z\" (UID: \"86a1f531-5f82-4b77-b776-762ae3ff6170\") " pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:45.072811 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.072785    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6rgv\" (UniqueName: \"kubernetes.io/projected/11a0b6c6-aae7-459a-9463-95459fc8ab61-kube-api-access-p6rgv\") pod \"node-ca-l6bw8\" (UID: \"11a0b6c6-aae7-459a-9463-95459fc8ab61\") " pod="openshift-image-registry/node-ca-l6bw8"
Jul 13 19:21:45.072972 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.072955    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pqgj\" (UniqueName: \"kubernetes.io/projected/9d300bbe-5196-46b2-96a3-a35f690f0207-kube-api-access-8pqgj\") pod \"node-resolver-6kk2c\" (UID: \"9d300bbe-5196-46b2-96a3-a35f690f0207\") " pod="openshift-dns/node-resolver-6kk2c"
Jul 13 19:21:45.073535 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.073507    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxhx9\" (UniqueName: \"kubernetes.io/projected/fff12175-e77b-4ec7-b0ca-eb4b45145f76-kube-api-access-zxhx9\") pod \"network-metrics-daemon-hjx4z\" (UID: \"fff12175-e77b-4ec7-b0ca-eb4b45145f76\") " pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:21:45.145908 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.145829    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zgxjr"
Jul 13 19:21:45.154918 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.154890    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7"
Jul 13 19:21:45.155215 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:45.155181    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd389114a_4b4f_42f5_8d8c_8dc751a82553.slice/crio-d5695971abdaed4a6692cdb8e9232f5d69c0e2c672b5b9bb7d725a1015bd19d7 WatchSource:0}: Error finding container d5695971abdaed4a6692cdb8e9232f5d69c0e2c672b5b9bb7d725a1015bd19d7: Status 404 returned error can't find the container with id d5695971abdaed4a6692cdb8e9232f5d69c0e2c672b5b9bb7d725a1015bd19d7
Jul 13 19:21:45.160913 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.160890    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wsf2z"
Jul 13 19:21:45.164841 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.164817    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7gssd"
Jul 13 19:21:45.165128 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:45.165095    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfad50d8c_d894_4a4d_a033_7ff1feb80dea.slice/crio-91d45e03403d2036fe5efe16ab61200fba2923ffb04ad613ed19131af0d845db WatchSource:0}: Error finding container 91d45e03403d2036fe5efe16ab61200fba2923ffb04ad613ed19131af0d845db: Status 404 returned error can't find the container with id 91d45e03403d2036fe5efe16ab61200fba2923ffb04ad613ed19131af0d845db
Jul 13 19:21:45.170081 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.170059    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xdp7v"
Jul 13 19:21:45.170478 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:45.170451    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86a1f531_5f82_4b77_b776_762ae3ff6170.slice/crio-b5abfe38bf138cdbc58b731fccb595f757f57f728753f8477261b7d8a25bfd22 WatchSource:0}: Error finding container b5abfe38bf138cdbc58b731fccb595f757f57f728753f8477261b7d8a25bfd22: Status 404 returned error can't find the container with id b5abfe38bf138cdbc58b731fccb595f757f57f728753f8477261b7d8a25bfd22
Jul 13 19:21:45.173529 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:45.173504    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc244587_d876_4024_8165_a2727f5d0863.slice/crio-a56f7efadaf98c0755c083cded8aecb6d424c8c57d9466a378e6df8e6db5d368 WatchSource:0}: Error finding container a56f7efadaf98c0755c083cded8aecb6d424c8c57d9466a378e6df8e6db5d368: Status 404 returned error can't find the container with id a56f7efadaf98c0755c083cded8aecb6d424c8c57d9466a378e6df8e6db5d368
Jul 13 19:21:45.174883 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.174864    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5dvcw"
Jul 13 19:21:45.180478 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.180459    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-l6bw8"
Jul 13 19:21:45.180796 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:45.180768    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15ae0b99_58a3_4072_beb3_abe52af62c10.slice/crio-448164744c72c38337bc93b074188292516a6e658b6426c09ecfa979339cdf5a WatchSource:0}: Error finding container 448164744c72c38337bc93b074188292516a6e658b6426c09ecfa979339cdf5a: Status 404 returned error can't find the container with id 448164744c72c38337bc93b074188292516a6e658b6426c09ecfa979339cdf5a
Jul 13 19:21:45.184205 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:45.184180    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2971001d_8014_4210_aa64_f477c4062715.slice/crio-18ff23362b0dd3161a09ad43cd15832fafe226219707def69406a4b0036361fc WatchSource:0}: Error finding container 18ff23362b0dd3161a09ad43cd15832fafe226219707def69406a4b0036361fc: Status 404 returned error can't find the container with id 18ff23362b0dd3161a09ad43cd15832fafe226219707def69406a4b0036361fc
Jul 13 19:21:45.185453 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.185386    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:21:45.189000 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.188977    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6kk2c"
Jul 13 19:21:45.190175 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:45.190152    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11a0b6c6_aae7_459a_9463_95459fc8ab61.slice/crio-e5a4f733aef4ab694637ff4c83aa3b1b5e8655e12ddec36e68a614febcdd6f0c WatchSource:0}: Error finding container e5a4f733aef4ab694637ff4c83aa3b1b5e8655e12ddec36e68a614febcdd6f0c: Status 404 returned error can't find the container with id e5a4f733aef4ab694637ff4c83aa3b1b5e8655e12ddec36e68a614febcdd6f0c
Jul 13 19:21:45.196920 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:45.196893    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d0d821f_0a58_4566_af42_eca2019fa233.slice/crio-d967ab72566a95bbc1015f64932a88969354b228f7eb314fa478327b7511a7f2 WatchSource:0}: Error finding container d967ab72566a95bbc1015f64932a88969354b228f7eb314fa478327b7511a7f2: Status 404 returned error can't find the container with id d967ab72566a95bbc1015f64932a88969354b228f7eb314fa478327b7511a7f2
Jul 13 19:21:45.200820 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:21:45.200792    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d300bbe_5196_46b2_96a3_a35f690f0207.slice/crio-519a31e792fd74651c080643ee7bd51a4c880c919847f80c58dd209772641073 WatchSource:0}: Error finding container 519a31e792fd74651c080643ee7bd51a4c880c919847f80c58dd209772641073: Status 404 returned error can't find the container with id 519a31e792fd74651c080643ee7bd51a4c880c919847f80c58dd209772641073
Jul 13 19:21:45.343453 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.343425    2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 19:21:45.462039 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.461441    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret\") pod \"global-pull-secret-syncer-xxrt7\" (UID: \"690fc787-2061-47e9-8a05-7e596f4ddf0d\") " pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:21:45.462039 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:45.461622    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 19:21:45.462039 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:45.461701    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret podName:690fc787-2061-47e9-8a05-7e596f4ddf0d nodeName:}" failed. No retries permitted until 2026-07-13 19:21:46.461680788 +0000 UTC m=+3.975557376 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret") pod "global-pull-secret-syncer-xxrt7" (UID: "690fc787-2061-47e9-8a05-7e596f4ddf0d") : object "kube-system"/"original-pull-secret" not registered
Jul 13 19:21:45.562737 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.562652    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs\") pod \"network-metrics-daemon-hjx4z\" (UID: \"fff12175-e77b-4ec7-b0ca-eb4b45145f76\") " pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:21:45.562737 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.562730    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlxgj\" (UniqueName: \"kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj\") pod \"network-check-target-2q22b\" (UID: \"cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8\") " pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:21:45.562993 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:45.562767    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:21:45.562993 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:45.562851    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs podName:fff12175-e77b-4ec7-b0ca-eb4b45145f76 nodeName:}" failed. No retries permitted until 2026-07-13 19:21:46.562832049 +0000 UTC m=+4.076708633 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs") pod "network-metrics-daemon-hjx4z" (UID: "fff12175-e77b-4ec7-b0ca-eb4b45145f76") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:21:45.562993 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:45.562874    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 19:21:45.562993 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:45.562892    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 19:21:45.562993 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:45.562905    2573 projected.go:196] Error preparing data for projected volume kube-api-access-dlxgj for pod openshift-network-diagnostics/network-check-target-2q22b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:21:45.562993 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:45.562951    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj podName:cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8 nodeName:}" failed. No retries permitted until 2026-07-13 19:21:46.56293661 +0000 UTC m=+4.076813201 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dlxgj" (UniqueName: "kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj") pod "network-check-target-2q22b" (UID: "cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:21:45.908744 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.908673    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 19:16:44 +0000 UTC" deadline="2028-04-28 22:08:04.695023442 +0000 UTC"
Jul 13 19:21:45.908744 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:45.908738    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15722h46m18.786289923s"
Jul 13 19:21:46.043348 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:46.042815    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:21:46.043348 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:46.042963    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxrt7" podUID="690fc787-2061-47e9-8a05-7e596f4ddf0d"
Jul 13 19:21:46.043348 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:46.043067    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:21:46.043348 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:46.043155    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjx4z" podUID="fff12175-e77b-4ec7-b0ca-eb4b45145f76"
Jul 13 19:21:46.043348 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:46.043207    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:21:46.043348 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:46.043274    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2q22b" podUID="cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8"
Jul 13 19:21:46.052299 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:46.052217    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6kk2c" event={"ID":"9d300bbe-5196-46b2-96a3-a35f690f0207","Type":"ContainerStarted","Data":"519a31e792fd74651c080643ee7bd51a4c880c919847f80c58dd209772641073"}
Jul 13 19:21:46.053934 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:46.053863    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-l6bw8" event={"ID":"11a0b6c6-aae7-459a-9463-95459fc8ab61","Type":"ContainerStarted","Data":"e5a4f733aef4ab694637ff4c83aa3b1b5e8655e12ddec36e68a614febcdd6f0c"}
Jul 13 19:21:46.057097 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:46.057018    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5dvcw" event={"ID":"2971001d-8014-4210-aa64-f477c4062715","Type":"ContainerStarted","Data":"18ff23362b0dd3161a09ad43cd15832fafe226219707def69406a4b0036361fc"}
Jul 13 19:21:46.058734 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:46.058708    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7gssd" event={"ID":"bc244587-d876-4024-8165-a2727f5d0863","Type":"ContainerStarted","Data":"a56f7efadaf98c0755c083cded8aecb6d424c8c57d9466a378e6df8e6db5d368"}
Jul 13 19:21:46.061629 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:46.061579    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wsf2z" event={"ID":"86a1f531-5f82-4b77-b776-762ae3ff6170","Type":"ContainerStarted","Data":"b5abfe38bf138cdbc58b731fccb595f757f57f728753f8477261b7d8a25bfd22"}
Jul 13 19:21:46.062956 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:46.062932    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7" event={"ID":"fad50d8c-d894-4a4d-a033-7ff1feb80dea","Type":"ContainerStarted","Data":"91d45e03403d2036fe5efe16ab61200fba2923ffb04ad613ed19131af0d845db"}
Jul 13 19:21:46.065526 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:46.065493    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5" event={"ID":"3d0d821f-0a58-4566-af42-eca2019fa233","Type":"ContainerStarted","Data":"d967ab72566a95bbc1015f64932a88969354b228f7eb314fa478327b7511a7f2"}
Jul 13 19:21:46.067903 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:46.067878    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xdp7v" event={"ID":"15ae0b99-58a3-4072-beb3-abe52af62c10","Type":"ContainerStarted","Data":"448164744c72c38337bc93b074188292516a6e658b6426c09ecfa979339cdf5a"}
Jul 13 19:21:46.070810 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:46.070741    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zgxjr" event={"ID":"d389114a-4b4f-42f5-8d8c-8dc751a82553","Type":"ContainerStarted","Data":"d5695971abdaed4a6692cdb8e9232f5d69c0e2c672b5b9bb7d725a1015bd19d7"}
Jul 13 19:21:46.470661 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:46.470261    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret\") pod \"global-pull-secret-syncer-xxrt7\" (UID: \"690fc787-2061-47e9-8a05-7e596f4ddf0d\") " pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:21:46.470661 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:46.470437    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 19:21:46.470661 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:46.470500    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret podName:690fc787-2061-47e9-8a05-7e596f4ddf0d nodeName:}" failed. No retries permitted until 2026-07-13 19:21:48.470481824 +0000 UTC m=+5.984358407 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret") pod "global-pull-secret-syncer-xxrt7" (UID: "690fc787-2061-47e9-8a05-7e596f4ddf0d") : object "kube-system"/"original-pull-secret" not registered
Jul 13 19:21:46.571330 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:46.571292    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs\") pod \"network-metrics-daemon-hjx4z\" (UID: \"fff12175-e77b-4ec7-b0ca-eb4b45145f76\") " pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:21:46.571512 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:46.571363    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlxgj\" (UniqueName: \"kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj\") pod \"network-check-target-2q22b\" (UID: \"cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8\") " pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:21:46.571577 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:46.571532    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 19:21:46.571577 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:46.571553    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 19:21:46.571577 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:46.571566    2573 projected.go:196] Error preparing data for projected volume kube-api-access-dlxgj for pod openshift-network-diagnostics/network-check-target-2q22b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:21:46.571738 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:46.571626    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj podName:cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8 nodeName:}" failed. No retries permitted until 2026-07-13 19:21:48.571606637 +0000 UTC m=+6.085483244 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dlxgj" (UniqueName: "kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj") pod "network-check-target-2q22b" (UID: "cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:21:46.572063 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:46.572041    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:21:46.572167 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:46.572097    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs podName:fff12175-e77b-4ec7-b0ca-eb4b45145f76 nodeName:}" failed. No retries permitted until 2026-07-13 19:21:48.572082725 +0000 UTC m=+6.085959310 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs") pod "network-metrics-daemon-hjx4z" (UID: "fff12175-e77b-4ec7-b0ca-eb4b45145f76") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:21:46.909866 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:46.909737    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 19:16:44 +0000 UTC" deadline="2027-12-20 13:46:36.942875759 +0000 UTC"
Jul 13 19:21:46.909866 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:46.909777    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12594h24m50.033103109s"
Jul 13 19:21:48.042143 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:48.042104    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:21:48.042628 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:48.042252    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxrt7" podUID="690fc787-2061-47e9-8a05-7e596f4ddf0d"
Jul 13 19:21:48.042730 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:48.042704    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:21:48.042848 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:48.042800    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2q22b" podUID="cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8"
Jul 13 19:21:48.042983 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:48.042651    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:21:48.043120 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:48.043099    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjx4z" podUID="fff12175-e77b-4ec7-b0ca-eb4b45145f76"
Jul 13 19:21:48.486704 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:48.486436    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret\") pod \"global-pull-secret-syncer-xxrt7\" (UID: \"690fc787-2061-47e9-8a05-7e596f4ddf0d\") " pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:21:48.486901 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:48.486616    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 19:21:48.486901 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:48.486851    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret podName:690fc787-2061-47e9-8a05-7e596f4ddf0d nodeName:}" failed. No retries permitted until 2026-07-13 19:21:52.486829065 +0000 UTC m=+10.000705696 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret") pod "global-pull-secret-syncer-xxrt7" (UID: "690fc787-2061-47e9-8a05-7e596f4ddf0d") : object "kube-system"/"original-pull-secret" not registered
Jul 13 19:21:48.588428 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:48.587585    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlxgj\" (UniqueName: \"kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj\") pod \"network-check-target-2q22b\" (UID: \"cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8\") " pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:21:48.588428 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:48.587690    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs\") pod \"network-metrics-daemon-hjx4z\" (UID: \"fff12175-e77b-4ec7-b0ca-eb4b45145f76\") " pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:21:48.588428 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:48.587825    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:21:48.588428 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:48.587894    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs podName:fff12175-e77b-4ec7-b0ca-eb4b45145f76 nodeName:}" failed. No retries permitted until 2026-07-13 19:21:52.587872631 +0000 UTC m=+10.101749238 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs") pod "network-metrics-daemon-hjx4z" (UID: "fff12175-e77b-4ec7-b0ca-eb4b45145f76") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:21:48.588428 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:48.588297    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 19:21:48.588428 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:48.588315    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 19:21:48.588428 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:48.588327    2573 projected.go:196] Error preparing data for projected volume kube-api-access-dlxgj for pod openshift-network-diagnostics/network-check-target-2q22b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:21:48.588428 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:48.588370    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj podName:cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8 nodeName:}" failed. No retries permitted until 2026-07-13 19:21:52.588355452 +0000 UTC m=+10.102232039 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dlxgj" (UniqueName: "kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj") pod "network-check-target-2q22b" (UID: "cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:21:50.043004 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:50.042894    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:21:50.043498 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:50.043045    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjx4z" podUID="fff12175-e77b-4ec7-b0ca-eb4b45145f76"
Jul 13 19:21:50.043498 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:50.043432    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:21:50.043612 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:50.043534    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2q22b" podUID="cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8"
Jul 13 19:21:50.043612 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:50.043584    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:21:50.043814 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:50.043681    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxrt7" podUID="690fc787-2061-47e9-8a05-7e596f4ddf0d"
Jul 13 19:21:52.042219 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:52.042181    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:21:52.042714 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:52.042331    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjx4z" podUID="fff12175-e77b-4ec7-b0ca-eb4b45145f76"
Jul 13 19:21:52.042784 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:52.042746    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:21:52.042864 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:52.042838    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2q22b" podUID="cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8"
Jul 13 19:21:52.042919 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:52.042904    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:21:52.042993 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:52.042975    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxrt7" podUID="690fc787-2061-47e9-8a05-7e596f4ddf0d"
Jul 13 19:21:52.522052 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:52.522010    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret\") pod \"global-pull-secret-syncer-xxrt7\" (UID: \"690fc787-2061-47e9-8a05-7e596f4ddf0d\") " pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:21:52.522234 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:52.522158    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 19:21:52.522234 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:52.522231    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret podName:690fc787-2061-47e9-8a05-7e596f4ddf0d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:00.522211886 +0000 UTC m=+18.036088487 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret") pod "global-pull-secret-syncer-xxrt7" (UID: "690fc787-2061-47e9-8a05-7e596f4ddf0d") : object "kube-system"/"original-pull-secret" not registered
Jul 13 19:21:52.622987 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:52.622935    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlxgj\" (UniqueName: \"kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj\") pod \"network-check-target-2q22b\" (UID: \"cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8\") " pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:21:52.623150 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:52.623022    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs\") pod \"network-metrics-daemon-hjx4z\" (UID: \"fff12175-e77b-4ec7-b0ca-eb4b45145f76\") " pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:21:52.623150 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:52.623099    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 19:21:52.623150 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:52.623127    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 19:21:52.623150 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:52.623142    2573 projected.go:196] Error preparing data for projected volume kube-api-access-dlxgj for pod openshift-network-diagnostics/network-check-target-2q22b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:21:52.623150 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:52.623149    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:21:52.623406 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:52.623205    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj podName:cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:00.623185661 +0000 UTC m=+18.137062260 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dlxgj" (UniqueName: "kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj") pod "network-check-target-2q22b" (UID: "cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:21:52.623406 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:52.623226    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs podName:fff12175-e77b-4ec7-b0ca-eb4b45145f76 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:00.623215711 +0000 UTC m=+18.137092300 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs") pod "network-metrics-daemon-hjx4z" (UID: "fff12175-e77b-4ec7-b0ca-eb4b45145f76") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:21:54.042763 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:54.042624    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:21:54.042763 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:54.042679    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:21:54.043234 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:54.042796    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjx4z" podUID="fff12175-e77b-4ec7-b0ca-eb4b45145f76"
Jul 13 19:21:54.043273 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:54.043228    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2q22b" podUID="cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8"
Jul 13 19:21:54.043315 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:54.043291    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:21:54.043383 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:54.043364    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxrt7" podUID="690fc787-2061-47e9-8a05-7e596f4ddf0d"
Jul 13 19:21:56.042838 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:56.042795    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:21:56.042838 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:56.042823    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:21:56.043225 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:56.042912    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2q22b" podUID="cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8"
Jul 13 19:21:56.043225 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:56.042964    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:21:56.043225 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:56.043046    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxrt7" podUID="690fc787-2061-47e9-8a05-7e596f4ddf0d"
Jul 13 19:21:56.043225 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:56.043117    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjx4z" podUID="fff12175-e77b-4ec7-b0ca-eb4b45145f76"
Jul 13 19:21:58.042211 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:58.042178    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:21:58.042681 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:58.042228    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:21:58.042681 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:21:58.042178    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:21:58.042681 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:58.042320    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2q22b" podUID="cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8"
Jul 13 19:21:58.042681 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:58.042423    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxrt7" podUID="690fc787-2061-47e9-8a05-7e596f4ddf0d"
Jul 13 19:21:58.042681 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:21:58.042517    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjx4z" podUID="fff12175-e77b-4ec7-b0ca-eb4b45145f76"
Jul 13 19:22:00.042486 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:00.042450    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:22:00.042486 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:00.042477    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:22:00.043155 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:00.042512    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:22:00.043155 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:00.042655    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxrt7" podUID="690fc787-2061-47e9-8a05-7e596f4ddf0d"
Jul 13 19:22:00.043155 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:00.042687    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2q22b" podUID="cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8"
Jul 13 19:22:00.043155 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:00.042783    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjx4z" podUID="fff12175-e77b-4ec7-b0ca-eb4b45145f76"
Jul 13 19:22:00.587549 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:00.587461    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret\") pod \"global-pull-secret-syncer-xxrt7\" (UID: \"690fc787-2061-47e9-8a05-7e596f4ddf0d\") " pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:22:00.587750 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:00.587581    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 19:22:00.587750 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:00.587659    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret podName:690fc787-2061-47e9-8a05-7e596f4ddf0d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:16.587624572 +0000 UTC m=+34.101501161 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret") pod "global-pull-secret-syncer-xxrt7" (UID: "690fc787-2061-47e9-8a05-7e596f4ddf0d") : object "kube-system"/"original-pull-secret" not registered
Jul 13 19:22:00.688654 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:00.688602    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs\") pod \"network-metrics-daemon-hjx4z\" (UID: \"fff12175-e77b-4ec7-b0ca-eb4b45145f76\") " pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:22:00.688845 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:00.688708    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlxgj\" (UniqueName: \"kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj\") pod \"network-check-target-2q22b\" (UID: \"cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8\") " pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:22:00.688845 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:00.688759    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:22:00.688845 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:00.688828    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs podName:fff12175-e77b-4ec7-b0ca-eb4b45145f76 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:16.688807982 +0000 UTC m=+34.202684585 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs") pod "network-metrics-daemon-hjx4z" (UID: "fff12175-e77b-4ec7-b0ca-eb4b45145f76") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:22:00.689021 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:00.688884    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 19:22:00.689021 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:00.688910    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 19:22:00.689021 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:00.688923    2573 projected.go:196] Error preparing data for projected volume kube-api-access-dlxgj for pod openshift-network-diagnostics/network-check-target-2q22b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:22:00.689021 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:00.688980    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj podName:cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:16.688962968 +0000 UTC m=+34.202839558 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dlxgj" (UniqueName: "kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj") pod "network-check-target-2q22b" (UID: "cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:22:02.042087 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:02.042047    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:22:02.042507 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:02.042135    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:22:02.042507 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:02.042264    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:22:02.042507 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:02.042283    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjx4z" podUID="fff12175-e77b-4ec7-b0ca-eb4b45145f76"
Jul 13 19:22:02.042507 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:02.042362    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2q22b" podUID="cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8"
Jul 13 19:22:02.042507 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:02.042442    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxrt7" podUID="690fc787-2061-47e9-8a05-7e596f4ddf0d"
Jul 13 19:22:04.042331 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:04.042290    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:22:04.042840 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:04.042291    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:22:04.042840 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:04.042429    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2q22b" podUID="cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8"
Jul 13 19:22:04.042840 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:04.042291    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:22:04.042840 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:04.042616    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjx4z" podUID="fff12175-e77b-4ec7-b0ca-eb4b45145f76"
Jul 13 19:22:04.042840 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:04.042489    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxrt7" podUID="690fc787-2061-47e9-8a05-7e596f4ddf0d"
Jul 13 19:22:05.113545 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:05.113286    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wsf2z" event={"ID":"86a1f531-5f82-4b77-b776-762ae3ff6170","Type":"ContainerStarted","Data":"7519fcaf4354f3827861c495d50bb98f386bf95de97564bb862c5b6cd5b09823"}
Jul 13 19:22:05.115173 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:05.115144    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7" event={"ID":"fad50d8c-d894-4a4d-a033-7ff1feb80dea","Type":"ContainerStarted","Data":"5511ddf8807b55ca548f00a174e317c9394dd511847e23ceaa5e5367161a3d46"}
Jul 13 19:22:05.116771 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:05.116741    2573 generic.go:356] "Generic (PLEG): container finished" podID="e34e84288ef00fb262466bd5864c1618" containerID="c8c39edee008add2be83c7395362212a3fe5084ecb71dd940febf1d859ac40a7" exitCode=0
Jul 13 19:22:05.116869 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:05.116825    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-91.ec2.internal" event={"ID":"e34e84288ef00fb262466bd5864c1618","Type":"ContainerDied","Data":"c8c39edee008add2be83c7395362212a3fe5084ecb71dd940febf1d859ac40a7"}
Jul 13 19:22:05.119074 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:05.119046    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-91.ec2.internal" event={"ID":"6db41158752585940a2ece7d4dc21de6","Type":"ContainerStarted","Data":"441c4aee7fc03ab094d2da9ca19c58e51773754487b592e2e08da97ab2a5d00f"}
Jul 13 19:22:05.122988 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:05.122971    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 19:22:05.123304 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:05.123286    2573 generic.go:356] "Generic (PLEG): container finished" podID="3d0d821f-0a58-4566-af42-eca2019fa233" containerID="317b68fd34dc3fd37c8addc03188899b3933d577647b3c3fc2bda1c3afd83ed8" exitCode=1
Jul 13 19:22:05.123373 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:05.123345    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5" event={"ID":"3d0d821f-0a58-4566-af42-eca2019fa233","Type":"ContainerStarted","Data":"7989f0f21c0a415988918868135346d9d828a1b90c268973667e15087a489297"}
Jul 13 19:22:05.123433 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:05.123408    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5" event={"ID":"3d0d821f-0a58-4566-af42-eca2019fa233","Type":"ContainerDied","Data":"317b68fd34dc3fd37c8addc03188899b3933d577647b3c3fc2bda1c3afd83ed8"}
Jul 13 19:22:05.123477 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:05.123453    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5" event={"ID":"3d0d821f-0a58-4566-af42-eca2019fa233","Type":"ContainerStarted","Data":"7dec2d49e87cfa9117e1dd391e78ddd7fdbaa5c9f4ef38a6453c92e5c888242e"}
Jul 13 19:22:05.124581 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:05.124562    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xdp7v" event={"ID":"15ae0b99-58a3-4072-beb3-abe52af62c10","Type":"ContainerStarted","Data":"1cb94c0e3970b8a6bd621a78deb61f1d865da60ed7a8ad8bba81a06ab02c582a"}
Jul 13 19:22:05.125900 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:05.125874    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zgxjr" event={"ID":"d389114a-4b4f-42f5-8d8c-8dc751a82553","Type":"ContainerStarted","Data":"dcb5adf3caf826bc1c4554646bee4f92f2b351b5235812a1b2e9d8d372a3d4dd"}
Jul 13 19:22:05.127274 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:05.127250    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6kk2c" event={"ID":"9d300bbe-5196-46b2-96a3-a35f690f0207","Type":"ContainerStarted","Data":"81693d0e7ed541d764ffb3d68fc0f569a1a7c1af421ebcdc7470c2632eddeb8f"}
Jul 13 19:22:05.128621 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:05.128602    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-l6bw8" event={"ID":"11a0b6c6-aae7-459a-9463-95459fc8ab61","Type":"ContainerStarted","Data":"d4dac7131a9056b13c61d3e53f8e26ccfc871a75b6c3332d51f63e3740ccbb3e"}
Jul 13 19:22:05.130405 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:05.129963    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5dvcw" event={"ID":"2971001d-8014-4210-aa64-f477c4062715","Type":"ContainerStarted","Data":"d75ea0dd04ac16c93a9d1ba6a9481adc4148c9768e7418cbd7a789586fe34a4c"}
Jul 13 19:22:05.156192 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:05.156047    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5dvcw" podStartSLOduration=2.5301673940000002 podStartE2EDuration="22.156031513s" totalImagesPullingTime="19.625864119s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:05.155453272 +0000 UTC m=+22.669329878" watchObservedRunningTime="2026-07-13 19:22:05.156031513 +0000 UTC m=+22.669908119"
Jul 13 19:22:05.187918 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:05.187875    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-l6bw8" podStartSLOduration=2.808203739 podStartE2EDuration="22.187857852s" totalImagesPullingTime="19.379654113s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:05.171518642 +0000 UTC m=+22.685395249" watchObservedRunningTime="2026-07-13 19:22:05.187857852 +0000 UTC m=+22.701734457"
Jul 13 19:22:05.188310 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:05.188287    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6kk2c" podStartSLOduration=2.77409233 podStartE2EDuration="22.188277958s" totalImagesPullingTime="19.414185628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:05.187769816 +0000 UTC m=+22.701646421" watchObservedRunningTime="2026-07-13 19:22:05.188277958 +0000 UTC m=+22.702154565"
Jul 13 19:22:05.230705 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:05.230661    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-91.ec2.internal" podStartSLOduration=21.230630609 podStartE2EDuration="21.230630609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:05.23060781 +0000 UTC m=+22.744484410" watchObservedRunningTime="2026-07-13 19:22:05.230630609 +0000 UTC m=+22.744507214"
Jul 13 19:22:05.282769 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:05.282714    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xdp7v" podStartSLOduration=7.136078268 podStartE2EDuration="22.282697507s" totalImagesPullingTime="15.146619239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:21:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:05.252618794 +0000 UTC m=+22.766495401" watchObservedRunningTime="2026-07-13 19:22:05.282697507 +0000 UTC m=+22.796574110"
Jul 13 19:22:06.042185 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:06.042140    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:22:06.042185 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:06.042169    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:22:06.042425 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:06.042154    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:22:06.042425 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:06.042257    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2q22b" podUID="cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8"
Jul 13 19:22:06.042425 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:06.042324    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjx4z" podUID="fff12175-e77b-4ec7-b0ca-eb4b45145f76"
Jul 13 19:22:06.042546 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:06.042425    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxrt7" podUID="690fc787-2061-47e9-8a05-7e596f4ddf0d"
Jul 13 19:22:06.136861 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:06.136834    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 19:22:06.137335 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:06.137202    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5" event={"ID":"3d0d821f-0a58-4566-af42-eca2019fa233","Type":"ContainerStarted","Data":"08cc5127d9055619fc50a1d35560f265e830ebf69e23850de41345065b1bf34d"}
Jul 13 19:22:06.137335 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:06.137276    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5" event={"ID":"3d0d821f-0a58-4566-af42-eca2019fa233","Type":"ContainerStarted","Data":"a609e1f6eb047a6536c1d290247b08a34a13b7c55b76c7d5567021ec13e3547c"}
Jul 13 19:22:06.137335 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:06.137320    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5" event={"ID":"3d0d821f-0a58-4566-af42-eca2019fa233","Type":"ContainerStarted","Data":"cff2610fe96f6d641d90bad72727dc13af5388635fa9d8ab75ff23aa2aafc71d"}
Jul 13 19:22:06.138570 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:06.138546    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7gssd" event={"ID":"bc244587-d876-4024-8165-a2727f5d0863","Type":"ContainerStarted","Data":"1656ec1f76d9ad0bea18e9848167747e8fd64b82ccb09282282d08ad420a0e8f"}
Jul 13 19:22:06.140152 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:06.140121    2573 generic.go:356] "Generic (PLEG): container finished" podID="86a1f531-5f82-4b77-b776-762ae3ff6170" containerID="7519fcaf4354f3827861c495d50bb98f386bf95de97564bb862c5b6cd5b09823" exitCode=0
Jul 13 19:22:06.140279 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:06.140199    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wsf2z" event={"ID":"86a1f531-5f82-4b77-b776-762ae3ff6170","Type":"ContainerDied","Data":"7519fcaf4354f3827861c495d50bb98f386bf95de97564bb862c5b6cd5b09823"}
Jul 13 19:22:06.142258 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:06.142237    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-91.ec2.internal" event={"ID":"e34e84288ef00fb262466bd5864c1618","Type":"ContainerStarted","Data":"350fc6d784c7fe86535486b8f70b649d8e979303f640c9d039c24f9123b2c452"}
Jul 13 19:22:06.155032 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:06.154988    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7gssd" podStartSLOduration=3.716429323 podStartE2EDuration="23.154972937s" totalImagesPullingTime="19.438543614s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:06.15479498 +0000 UTC m=+23.668671588" watchObservedRunningTime="2026-07-13 19:22:06.154972937 +0000 UTC m=+23.668849589"
Jul 13 19:22:06.155179 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:06.155118    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zgxjr" podStartSLOduration=3.696120111 podStartE2EDuration="23.155111672s" totalImagesPullingTime="19.458991561s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:05.28283922 +0000 UTC m=+22.796715818" watchObservedRunningTime="2026-07-13 19:22:06.155111672 +0000 UTC m=+23.668988296"
Jul 13 19:22:06.200384 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:06.200336    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-91.ec2.internal" podStartSLOduration=22.200321013 podStartE2EDuration="22.200321013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:06.200188889 +0000 UTC m=+23.714065495" watchObservedRunningTime="2026-07-13 19:22:06.200321013 +0000 UTC m=+23.714197618"
Jul 13 19:22:06.420244 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:06.420210    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xdp7v"
Jul 13 19:22:06.420796 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:06.420777    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xdp7v"
Jul 13 19:22:07.144916 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:07.144886    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xdp7v"
Jul 13 19:22:07.145415 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:07.145393    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xdp7v"
Jul 13 19:22:07.177190 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:07.177170    2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 13 19:22:07.910626 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:07.910253    2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T19:22:07.177187065Z","UUID":"edd95005-54e1-493d-bc5f-2bfe1d52e7d4","Handler":null,"Name":"","Endpoint":""}
Jul 13 19:22:07.912233 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:07.912210    2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 13 19:22:07.912382 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:07.912277    2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 13 19:22:08.042100 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:08.042055    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:22:08.042100 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:08.042084    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:22:08.042344 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:08.042054    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:22:08.042344 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:08.042168    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxrt7" podUID="690fc787-2061-47e9-8a05-7e596f4ddf0d"
Jul 13 19:22:08.042344 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:08.042263    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2q22b" podUID="cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8"
Jul 13 19:22:08.042470 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:08.042367    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjx4z" podUID="fff12175-e77b-4ec7-b0ca-eb4b45145f76"
Jul 13 19:22:08.148897 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:08.148858    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7" event={"ID":"fad50d8c-d894-4a4d-a033-7ff1feb80dea","Type":"ContainerStarted","Data":"e8208990422e5d366e0b8c61e566730b1283027900a6cce6cdc9646061520486"}
Jul 13 19:22:08.153098 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:08.153075    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 19:22:08.153628 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:08.153561    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5" event={"ID":"3d0d821f-0a58-4566-af42-eca2019fa233","Type":"ContainerStarted","Data":"8b413403ad44684b7e681cfe4ed8e405be8220344549551afabb1123e8977b4e"}
Jul 13 19:22:09.158234 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:09.158184    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7" event={"ID":"fad50d8c-d894-4a4d-a033-7ff1feb80dea","Type":"ContainerStarted","Data":"fca87079e33a11a5524775ce7906073a1172e89b44ef1f5be7df708c76053595"}
Jul 13 19:22:09.181488 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:09.181439    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kpqb7" podStartSLOduration=3.51555804 podStartE2EDuration="26.181422917s" totalImagesPullingTime="22.665864877s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:09.181019282 +0000 UTC m=+26.694895890" watchObservedRunningTime="2026-07-13 19:22:09.181422917 +0000 UTC m=+26.695299523"
Jul 13 19:22:10.042671 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:10.042612    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:22:10.042876 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:10.042617    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:22:10.042876 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:10.042761    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:22:10.042876 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:10.042769    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2q22b" podUID="cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8"
Jul 13 19:22:10.043043 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:10.042885    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjx4z" podUID="fff12175-e77b-4ec7-b0ca-eb4b45145f76"
Jul 13 19:22:10.043043 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:10.042959    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxrt7" podUID="690fc787-2061-47e9-8a05-7e596f4ddf0d"
Jul 13 19:22:11.164906 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:11.164651    2573 generic.go:356] "Generic (PLEG): container finished" podID="86a1f531-5f82-4b77-b776-762ae3ff6170" containerID="4bb9b924f095a0b5ac03f48ad3c4a3342efca836e046f971c600f1ed7eb53eaf" exitCode=0
Jul 13 19:22:11.165587 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:11.164722    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wsf2z" event={"ID":"86a1f531-5f82-4b77-b776-762ae3ff6170","Type":"ContainerDied","Data":"4bb9b924f095a0b5ac03f48ad3c4a3342efca836e046f971c600f1ed7eb53eaf"}
Jul 13 19:22:11.168456 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:11.168437    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 19:22:11.168823 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:11.168793    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5" event={"ID":"3d0d821f-0a58-4566-af42-eca2019fa233","Type":"ContainerStarted","Data":"990130bdfe2e2388b68597ad1d78ae09a26f27352caa93bbf7d7d5d1b9bdaada"}
Jul 13 19:22:11.169052 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:11.169033    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:22:11.169141 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:11.169110    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:22:11.169202 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:11.169159    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:22:11.169249 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:11.169218    2573 scope.go:117] "RemoveContainer" containerID="317b68fd34dc3fd37c8addc03188899b3933d577647b3c3fc2bda1c3afd83ed8"
Jul 13 19:22:11.185554 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:11.185531    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:22:11.186051 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:11.186024    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:22:12.042921 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:12.042888    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:22:12.043163 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:12.042887    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:22:12.043163 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:12.043010    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2q22b" podUID="cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8"
Jul 13 19:22:12.043163 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:12.042887    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:22:12.043163 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:12.043090    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjx4z" podUID="fff12175-e77b-4ec7-b0ca-eb4b45145f76"
Jul 13 19:22:12.043338 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:12.043186    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxrt7" podUID="690fc787-2061-47e9-8a05-7e596f4ddf0d"
Jul 13 19:22:12.174657 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:12.174616    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 19:22:12.175050 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:12.175021    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5" event={"ID":"3d0d821f-0a58-4566-af42-eca2019fa233","Type":"ContainerStarted","Data":"b26ec7630ada8a23446da18739c61ac9017ae8e04e025acf7c03bb3a70aaf970"}
Jul 13 19:22:12.209357 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:12.209312    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5" podStartSLOduration=9.772831322 podStartE2EDuration="29.209295571s" totalImagesPullingTime="19.436464249s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:12.207627261 +0000 UTC m=+29.721503885" watchObservedRunningTime="2026-07-13 19:22:12.209295571 +0000 UTC m=+29.723172177"
Jul 13 19:22:12.233225 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:12.233197    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xxrt7"]
Jul 13 19:22:12.233372 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:12.233294    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:22:12.233424 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:12.233379    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxrt7" podUID="690fc787-2061-47e9-8a05-7e596f4ddf0d"
Jul 13 19:22:12.238231 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:12.238206    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2q22b"]
Jul 13 19:22:12.238333 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:12.238290    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:22:12.238375 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:12.238362    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2q22b" podUID="cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8"
Jul 13 19:22:12.241794 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:12.241771    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hjx4z"]
Jul 13 19:22:12.241911 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:12.241875    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:22:12.242002 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:12.241982    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjx4z" podUID="fff12175-e77b-4ec7-b0ca-eb4b45145f76"
Jul 13 19:22:13.179237 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:13.179202    2573 generic.go:356] "Generic (PLEG): container finished" podID="86a1f531-5f82-4b77-b776-762ae3ff6170" containerID="ff7030e1a1d39fafde24a39f18bb06c8ea76b72478c483d892a4a12af3ba3933" exitCode=0
Jul 13 19:22:13.179630 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:13.179292    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wsf2z" event={"ID":"86a1f531-5f82-4b77-b776-762ae3ff6170","Type":"ContainerDied","Data":"ff7030e1a1d39fafde24a39f18bb06c8ea76b72478c483d892a4a12af3ba3933"}
Jul 13 19:22:14.043054 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:14.042765    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:22:14.043190 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:14.042775    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:22:14.043190 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:14.043122    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjx4z" podUID="fff12175-e77b-4ec7-b0ca-eb4b45145f76"
Jul 13 19:22:14.043190 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:14.043169    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxrt7" podUID="690fc787-2061-47e9-8a05-7e596f4ddf0d"
Jul 13 19:22:14.043190 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:14.042810    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:22:14.043345 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:14.043232    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2q22b" podUID="cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8"
Jul 13 19:22:15.186032 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:15.185996    2573 generic.go:356] "Generic (PLEG): container finished" podID="86a1f531-5f82-4b77-b776-762ae3ff6170" containerID="c34a389a058a792d7c8532a0a060ad8dbf47f59caf360b0ef3a2bd47ab85ad87" exitCode=0
Jul 13 19:22:15.186032 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:15.186041    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wsf2z" event={"ID":"86a1f531-5f82-4b77-b776-762ae3ff6170","Type":"ContainerDied","Data":"c34a389a058a792d7c8532a0a060ad8dbf47f59caf360b0ef3a2bd47ab85ad87"}
Jul 13 19:22:16.041998 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:16.041961    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:22:16.041998 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:16.041990    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:22:16.042255 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:16.041993    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:22:16.042255 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:16.042109    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjx4z" podUID="fff12175-e77b-4ec7-b0ca-eb4b45145f76"
Jul 13 19:22:16.042255 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:16.042194    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2q22b" podUID="cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8"
Jul 13 19:22:16.042401 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:16.042289    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxrt7" podUID="690fc787-2061-47e9-8a05-7e596f4ddf0d"
Jul 13 19:22:16.602835 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:16.602800    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret\") pod \"global-pull-secret-syncer-xxrt7\" (UID: \"690fc787-2061-47e9-8a05-7e596f4ddf0d\") " pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:22:16.603278 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:16.602933    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 19:22:16.603278 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:16.602996    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret podName:690fc787-2061-47e9-8a05-7e596f4ddf0d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:48.602980541 +0000 UTC m=+66.116857125 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret") pod "global-pull-secret-syncer-xxrt7" (UID: "690fc787-2061-47e9-8a05-7e596f4ddf0d") : object "kube-system"/"original-pull-secret" not registered
Jul 13 19:22:16.703500 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:16.703465    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs\") pod \"network-metrics-daemon-hjx4z\" (UID: \"fff12175-e77b-4ec7-b0ca-eb4b45145f76\") " pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:22:16.703723 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:16.703519    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlxgj\" (UniqueName: \"kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj\") pod \"network-check-target-2q22b\" (UID: \"cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8\") " pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:22:16.703723 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:16.703620    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:22:16.703723 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:16.703661    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 19:22:16.703723 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:16.703679    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 19:22:16.703723 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:16.703692    2573 projected.go:196] Error preparing data for projected volume kube-api-access-dlxgj for pod openshift-network-diagnostics/network-check-target-2q22b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:22:16.703963 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:16.703710    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs podName:fff12175-e77b-4ec7-b0ca-eb4b45145f76 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:48.703689697 +0000 UTC m=+66.217566295 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs") pod "network-metrics-daemon-hjx4z" (UID: "fff12175-e77b-4ec7-b0ca-eb4b45145f76") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 19:22:16.703963 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:16.703755    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj podName:cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:48.703745424 +0000 UTC m=+66.217622022 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dlxgj" (UniqueName: "kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj") pod "network-check-target-2q22b" (UID: "cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 19:22:18.042487 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.042446    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:22:18.042974 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.042446    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:22:18.042974 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:18.042592    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjx4z" podUID="fff12175-e77b-4ec7-b0ca-eb4b45145f76"
Jul 13 19:22:18.042974 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.042446    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:22:18.042974 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:18.042660    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2q22b" podUID="cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8"
Jul 13 19:22:18.042974 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:18.042714    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxrt7" podUID="690fc787-2061-47e9-8a05-7e596f4ddf0d"
Jul 13 19:22:18.313978 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.313886    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-91.ec2.internal" event="NodeReady"
Jul 13 19:22:18.314132 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.314091    2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 13 19:22:18.350783 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.350750    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b77547c98-zzq67"]
Jul 13 19:22:18.377064 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.374700    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-s9z7s"]
Jul 13 19:22:18.377064 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.374806    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.383497 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.383468    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 13 19:22:18.383679 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.383509    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 13 19:22:18.383679 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.383568    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 13 19:22:18.383790 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.383716    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-25765\""
Jul 13 19:22:18.389423 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.389395    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-c55st"]
Jul 13 19:22:18.389554 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.389536    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-s9z7s"
Jul 13 19:22:18.392425 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.392402    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 13 19:22:18.392713 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.392692    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mf9hk\""
Jul 13 19:22:18.393025 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.393004    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 13 19:22:18.393731 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.393378    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 13 19:22:18.400952 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.400083    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b77547c98-zzq67"]
Jul 13 19:22:18.400952 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.400126    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-mmdms"]
Jul 13 19:22:18.400952 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.400595    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-c55st"
Jul 13 19:22:18.403337 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.403236    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 13 19:22:18.403995 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.403972    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 13 19:22:18.405060 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.404891    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-fltxf\""
Jul 13 19:22:18.405060 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.404934    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 13 19:22:18.405243 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.405222    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 13 19:22:18.411738 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.411720    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 13 19:22:18.411855 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.411781    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvc57"]
Jul 13 19:22:18.411932 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.411913    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-mmdms"
Jul 13 19:22:18.415235 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.415212    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-64rh5\""
Jul 13 19:22:18.415336 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.415319    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 13 19:22:18.415985 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.415964    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 13 19:22:18.426990 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.426951    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-p47mr"]
Jul 13 19:22:18.427118 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.427070    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvc57"
Jul 13 19:22:18.429917 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.429898    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-vd9lq\""
Jul 13 19:22:18.430260 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.430234    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 13 19:22:18.430361 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.430328    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 13 19:22:18.431016 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.430995    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 13 19:22:18.431168 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.431125    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 13 19:22:18.437272 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.437254    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-dqtq2"]
Jul 13 19:22:18.437386 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.437371    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-p47mr"
Jul 13 19:22:18.439938 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.439916    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 13 19:22:18.440044 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.439944    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 13 19:22:18.440322 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.440306    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 13 19:22:18.440430 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.440305    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 13 19:22:18.440623 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.440573    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-5bngn\""
Jul 13 19:22:18.447971 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.447948    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7556r"]
Jul 13 19:22:18.448094 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.448079    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-dqtq2"
Jul 13 19:22:18.450522 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.450504    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 13 19:22:18.450934 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.450916    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 13 19:22:18.451045 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.450934    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-kqx5c\""
Jul 13 19:22:18.456812 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.456781    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-nnsb7"]
Jul 13 19:22:18.457326 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.456916    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7556r"
Jul 13 19:22:18.460244 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.460223    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 13 19:22:18.460244 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.460237    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 13 19:22:18.460386 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.460313    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-pz9r6\""
Jul 13 19:22:18.460476 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.460461    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 13 19:22:18.461850 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.461831    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-mmdms"]
Jul 13 19:22:18.461955 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.461858    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-p47mr"]
Jul 13 19:22:18.461955 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.461872    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-s9z7s"]
Jul 13 19:22:18.461955 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.461898    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-59rng"]
Jul 13 19:22:18.462107 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.461969    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7"
Jul 13 19:22:18.465401 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.465328    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 13 19:22:18.465401 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.465341    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 13 19:22:18.465401 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.465371    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 13 19:22:18.465401 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.465390    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-zg2x5\""
Jul 13 19:22:18.465659 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.465377    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 13 19:22:18.467925 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.467905    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-gg4gl"]
Jul 13 19:22:18.468047 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.468028    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-59rng"
Jul 13 19:22:18.470879 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.470858    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 13 19:22:18.471086 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.471071    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bvq9g\""
Jul 13 19:22:18.471159 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.471141    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 13 19:22:18.471239 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.471227    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 13 19:22:18.472011 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.471993    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 13 19:22:18.473821 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.473799    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb"]
Jul 13 19:22:18.474140 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.474121    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-gg4gl"
Jul 13 19:22:18.476715 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.476696    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 13 19:22:18.476805 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.476696    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 13 19:22:18.476805 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.476730    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-ltmhz\""
Jul 13 19:22:18.476996 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.476977    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-68dc6f48c8-qx2hp"]
Jul 13 19:22:18.477114 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.477099    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb"
Jul 13 19:22:18.479698 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.479675    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 13 19:22:18.479906 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.479883    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 13 19:22:18.480393 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.480375    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 13 19:22:18.480673 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.480623    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xr8pl\""
Jul 13 19:22:18.480882 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.480866    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 13 19:22:18.485196 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.485178    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-nnsb7"]
Jul 13 19:22:18.485307 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.485289    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7556r"]
Jul 13 19:22:18.485373 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.485323    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:18.485573 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.485328    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-c55st"]
Jul 13 19:22:18.485670 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.485612    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvc57"]
Jul 13 19:22:18.485670 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.485667    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-dqtq2"]
Jul 13 19:22:18.485778 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.485691    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-59rng"]
Jul 13 19:22:18.485778 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.485718    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb"]
Jul 13 19:22:18.485778 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.485743    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-gg4gl"]
Jul 13 19:22:18.485916 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.485788    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-68dc6f48c8-qx2hp"]
Jul 13 19:22:18.487534 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.487516    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 13 19:22:18.487685 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.487569    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 13 19:22:18.487917 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.487883    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 13 19:22:18.487992 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.487921    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 13 19:22:18.488135 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.488119    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-g8xzp\""
Jul 13 19:22:18.488257 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.488239    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 13 19:22:18.488475 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.488454    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 13 19:22:18.517765 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.517740    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b1572934-6111-4394-9114-396122b025e7-installation-pull-secrets\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.517917 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.517779    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-bound-sa-token\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.517917 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.517805    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c6m5\" (UniqueName: \"kubernetes.io/projected/319e53f6-7d9e-48f3-b9f6-001ada635739-kube-api-access-5c6m5\") pod \"console-operator-7866497494-c55st\" (UID: \"319e53f6-7d9e-48f3-b9f6-001ada635739\") " pod="openshift-console-operator/console-operator-7866497494-c55st"
Jul 13 19:22:18.517917 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.517869    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b1572934-6111-4394-9114-396122b025e7-ca-trust-extracted\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.517917 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.517900    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbgdv\" (UniqueName: \"kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-kube-api-access-zbgdv\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.518119 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.517926    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0fce8f4-676e-4fec-adf5-09b3fd26434d-tmp-dir\") pod \"dns-default-s9z7s\" (UID: \"a0fce8f4-676e-4fec-adf5-09b3fd26434d\") " pod="openshift-dns/dns-default-s9z7s"
Jul 13 19:22:18.518119 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.517952    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fce914fd-6bde-48a0-9215-e66d18b67607-serving-cert\") pod \"service-ca-operator-7695fc6c8c-hvc57\" (UID: \"fce914fd-6bde-48a0-9215-e66d18b67607\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvc57"
Jul 13 19:22:18.518119 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.518002    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/319e53f6-7d9e-48f3-b9f6-001ada635739-serving-cert\") pod \"console-operator-7866497494-c55st\" (UID: \"319e53f6-7d9e-48f3-b9f6-001ada635739\") " pod="openshift-console-operator/console-operator-7866497494-c55st"
Jul 13 19:22:18.518119 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.518039    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a0fce8f4-676e-4fec-adf5-09b3fd26434d-config-volume\") pod \"dns-default-s9z7s\" (UID: \"a0fce8f4-676e-4fec-adf5-09b3fd26434d\") " pod="openshift-dns/dns-default-s9z7s"
Jul 13 19:22:18.518119 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.518067    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m87k\" (UniqueName: \"kubernetes.io/projected/a0fce8f4-676e-4fec-adf5-09b3fd26434d-kube-api-access-5m87k\") pod \"dns-default-s9z7s\" (UID: \"a0fce8f4-676e-4fec-adf5-09b3fd26434d\") " pod="openshift-dns/dns-default-s9z7s"
Jul 13 19:22:18.518119 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.518106    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-p47mr\" (UID: \"f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-p47mr"
Jul 13 19:22:18.518353 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.518183    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/54bb1498-3f40-47ca-a33c-eaebcf72090b-nginx-conf\") pod \"networking-console-plugin-6cf856758c-dqtq2\" (UID: \"54bb1498-3f40-47ca-a33c-eaebcf72090b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-dqtq2"
Jul 13 19:22:18.518353 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.518220    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54bb1498-3f40-47ca-a33c-eaebcf72090b-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-dqtq2\" (UID: \"54bb1498-3f40-47ca-a33c-eaebcf72090b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-dqtq2"
Jul 13 19:22:18.518353 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.518249    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0fce8f4-676e-4fec-adf5-09b3fd26434d-metrics-tls\") pod \"dns-default-s9z7s\" (UID: \"a0fce8f4-676e-4fec-adf5-09b3fd26434d\") " pod="openshift-dns/dns-default-s9z7s"
Jul 13 19:22:18.518353 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.518276    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x7s6\" (UniqueName: \"kubernetes.io/projected/fce914fd-6bde-48a0-9215-e66d18b67607-kube-api-access-6x7s6\") pod \"service-ca-operator-7695fc6c8c-hvc57\" (UID: \"fce914fd-6bde-48a0-9215-e66d18b67607\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvc57"
Jul 13 19:22:18.518353 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.518305    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/319e53f6-7d9e-48f3-b9f6-001ada635739-config\") pod \"console-operator-7866497494-c55st\" (UID: \"319e53f6-7d9e-48f3-b9f6-001ada635739\") " pod="openshift-console-operator/console-operator-7866497494-c55st"
Jul 13 19:22:18.518353 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.518320    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jp7f\" (UniqueName: \"kubernetes.io/projected/507498fd-317d-48a0-b218-ea88bf7e6a58-kube-api-access-4jp7f\") pod \"volume-data-source-validator-78495597f6-mmdms\" (UID: \"507498fd-317d-48a0-b218-ea88bf7e6a58\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-mmdms"
Jul 13 19:22:18.518353 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.518339    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fce914fd-6bde-48a0-9215-e66d18b67607-config\") pod \"service-ca-operator-7695fc6c8c-hvc57\" (UID: \"fce914fd-6bde-48a0-9215-e66d18b67607\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvc57"
Jul 13 19:22:18.518579 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.518369    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b1572934-6111-4394-9114-396122b025e7-image-registry-private-configuration\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.518579 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.518387    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b1572934-6111-4394-9114-396122b025e7-registry-certificates\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.518579 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.518402    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/319e53f6-7d9e-48f3-b9f6-001ada635739-trusted-ca\") pod \"console-operator-7866497494-c55st\" (UID: \"319e53f6-7d9e-48f3-b9f6-001ada635739\") " pod="openshift-console-operator/console-operator-7866497494-c55st"
Jul 13 19:22:18.518579 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.518464    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-p47mr\" (UID: \"f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-p47mr"
Jul 13 19:22:18.518579 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.518484    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jrbh\" (UniqueName: \"kubernetes.io/projected/f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc-kube-api-access-8jrbh\") pod \"kube-storage-version-migrator-operator-799cb7f958-p47mr\" (UID: \"f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-p47mr"
Jul 13 19:22:18.518579 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.518500    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.518579 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.518531    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b1572934-6111-4394-9114-396122b025e7-trusted-ca\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.619111 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.619028    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95371626-7fce-48f7-a5ba-91cc14074e85-cert\") pod \"ingress-canary-59rng\" (UID: \"95371626-7fce-48f7-a5ba-91cc14074e85\") " pod="openshift-ingress-canary/ingress-canary-59rng"
Jul 13 19:22:18.619270 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.619125    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a55203b2-d840-4319-9ced-26799e123698-service-ca-bundle\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:18.619270 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.619204    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-p47mr\" (UID: \"f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-p47mr"
Jul 13 19:22:18.619270 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.619242    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/54bb1498-3f40-47ca-a33c-eaebcf72090b-nginx-conf\") pod \"networking-console-plugin-6cf856758c-dqtq2\" (UID: \"54bb1498-3f40-47ca-a33c-eaebcf72090b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-dqtq2"
Jul 13 19:22:18.619270 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.619261    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5945d50-bd96-4904-96ba-ea69aaef93d8-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-nnsb7\" (UID: \"c5945d50-bd96-4904-96ba-ea69aaef93d8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7"
Jul 13 19:22:18.619484 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.619281    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54bb1498-3f40-47ca-a33c-eaebcf72090b-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-dqtq2\" (UID: \"54bb1498-3f40-47ca-a33c-eaebcf72090b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-dqtq2"
Jul 13 19:22:18.619484 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.619305    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-stats-auth\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:18.619484 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.619330    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6x7s6\" (UniqueName: \"kubernetes.io/projected/fce914fd-6bde-48a0-9215-e66d18b67607-kube-api-access-6x7s6\") pod \"service-ca-operator-7695fc6c8c-hvc57\" (UID: \"fce914fd-6bde-48a0-9215-e66d18b67607\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvc57"
Jul 13 19:22:18.619484 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.619358    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/319e53f6-7d9e-48f3-b9f6-001ada635739-config\") pod \"console-operator-7866497494-c55st\" (UID: \"319e53f6-7d9e-48f3-b9f6-001ada635739\") " pod="openshift-console-operator/console-operator-7866497494-c55st"
Jul 13 19:22:18.619484 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.619381    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c5945d50-bd96-4904-96ba-ea69aaef93d8-snapshots\") pod \"insights-operator-68d9dd94cf-nnsb7\" (UID: \"c5945d50-bd96-4904-96ba-ea69aaef93d8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7"
Jul 13 19:22:18.619484 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.619407    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c0acc219-19c5-4387-a6d3-7c83cffb00df-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-gr7gb\" (UID: \"c0acc219-19c5-4387-a6d3-7c83cffb00df\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb"
Jul 13 19:22:18.619484 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.619445    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddf2l\" (UniqueName: \"kubernetes.io/projected/c0acc219-19c5-4387-a6d3-7c83cffb00df-kube-api-access-ddf2l\") pod \"cluster-monitoring-operator-5f7f494dd4-gr7gb\" (UID: \"c0acc219-19c5-4387-a6d3-7c83cffb00df\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb"
Jul 13 19:22:18.619484 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.619475    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b1572934-6111-4394-9114-396122b025e7-image-registry-private-configuration\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.619896 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:18.619500    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 13 19:22:18.619896 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.619508    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b1572934-6111-4394-9114-396122b025e7-registry-certificates\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.619896 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.619538    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c5945d50-bd96-4904-96ba-ea69aaef93d8-serving-cert\") pod \"insights-operator-68d9dd94cf-nnsb7\" (UID: \"c5945d50-bd96-4904-96ba-ea69aaef93d8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7"
Jul 13 19:22:18.619896 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.619558    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0acc219-19c5-4387-a6d3-7c83cffb00df-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gr7gb\" (UID: \"c0acc219-19c5-4387-a6d3-7c83cffb00df\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb"
Jul 13 19:22:18.619896 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:18.619578    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54bb1498-3f40-47ca-a33c-eaebcf72090b-networking-console-plugin-cert podName:54bb1498-3f40-47ca-a33c-eaebcf72090b nodeName:}" failed. No retries permitted until 2026-07-13 19:22:19.119556945 +0000 UTC m=+36.633433529 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/54bb1498-3f40-47ca-a33c-eaebcf72090b-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-dqtq2" (UID: "54bb1498-3f40-47ca-a33c-eaebcf72090b") : secret "networking-console-plugin-cert" not found
Jul 13 19:22:18.619896 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.619833    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-p47mr\" (UID: \"f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-p47mr"
Jul 13 19:22:18.619896 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.619869    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jrbh\" (UniqueName: \"kubernetes.io/projected/f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc-kube-api-access-8jrbh\") pod \"kube-storage-version-migrator-operator-799cb7f958-p47mr\" (UID: \"f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-p47mr"
Jul 13 19:22:18.620227 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.619895    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b1572934-6111-4394-9114-396122b025e7-trusted-ca\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.620227 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.620141    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/319e53f6-7d9e-48f3-b9f6-001ada635739-serving-cert\") pod \"console-operator-7866497494-c55st\" (UID: \"319e53f6-7d9e-48f3-b9f6-001ada635739\") " pod="openshift-console-operator/console-operator-7866497494-c55st"
Jul 13 19:22:18.620227 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.620174    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5945d50-bd96-4904-96ba-ea69aaef93d8-tmp\") pod \"insights-operator-68d9dd94cf-nnsb7\" (UID: \"c5945d50-bd96-4904-96ba-ea69aaef93d8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7"
Jul 13 19:22:18.620227 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.620203    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfhsz\" (UniqueName: \"kubernetes.io/projected/c5945d50-bd96-4904-96ba-ea69aaef93d8-kube-api-access-zfhsz\") pod \"insights-operator-68d9dd94cf-nnsb7\" (UID: \"c5945d50-bd96-4904-96ba-ea69aaef93d8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7"
Jul 13 19:22:18.620417 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.620231    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnqss\" (UniqueName: \"kubernetes.io/projected/a55203b2-d840-4319-9ced-26799e123698-kube-api-access-rnqss\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:18.620417 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.620260    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fce914fd-6bde-48a0-9215-e66d18b67607-config\") pod \"service-ca-operator-7695fc6c8c-hvc57\" (UID: \"fce914fd-6bde-48a0-9215-e66d18b67607\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvc57"
Jul 13 19:22:18.620417 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.620296    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.620417 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.619975    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-p47mr\" (UID: \"f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-p47mr"
Jul 13 19:22:18.620417 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.620086    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/54bb1498-3f40-47ca-a33c-eaebcf72090b-nginx-conf\") pod \"networking-console-plugin-6cf856758c-dqtq2\" (UID: \"54bb1498-3f40-47ca-a33c-eaebcf72090b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-dqtq2"
Jul 13 19:22:18.620417 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.620389    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/319e53f6-7d9e-48f3-b9f6-001ada635739-config\") pod \"console-operator-7866497494-c55st\" (UID: \"319e53f6-7d9e-48f3-b9f6-001ada635739\") " pod="openshift-console-operator/console-operator-7866497494-c55st"
Jul 13 19:22:18.620723 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:18.620482    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 19:22:18.620723 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:18.620498    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b77547c98-zzq67: secret "image-registry-tls" not found
Jul 13 19:22:18.620723 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:18.620549    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls podName:b1572934-6111-4394-9114-396122b025e7 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:19.120534742 +0000 UTC m=+36.634411325 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls") pod "image-registry-5b77547c98-zzq67" (UID: "b1572934-6111-4394-9114-396122b025e7") : secret "image-registry-tls" not found
Jul 13 19:22:18.620723 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.620592    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0fce8f4-676e-4fec-adf5-09b3fd26434d-tmp-dir\") pod \"dns-default-s9z7s\" (UID: \"a0fce8f4-676e-4fec-adf5-09b3fd26434d\") " pod="openshift-dns/dns-default-s9z7s"
Jul 13 19:22:18.620723 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.620626    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c6m5\" (UniqueName: \"kubernetes.io/projected/319e53f6-7d9e-48f3-b9f6-001ada635739-kube-api-access-5c6m5\") pod \"console-operator-7866497494-c55st\" (UID: \"319e53f6-7d9e-48f3-b9f6-001ada635739\") " pod="openshift-console-operator/console-operator-7866497494-c55st"
Jul 13 19:22:18.620723 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.620677    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbgdv\" (UniqueName: \"kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-kube-api-access-zbgdv\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.620723 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.620705    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-metrics-certs\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:18.621044 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.620734    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j447v\" (UniqueName: \"kubernetes.io/projected/51a39015-504a-4953-82e0-45eb6b25be9d-kube-api-access-j447v\") pod \"cluster-samples-operator-65db858894-7556r\" (UID: \"51a39015-504a-4953-82e0-45eb6b25be9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7556r"
Jul 13 19:22:18.621044 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.620764    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a0fce8f4-676e-4fec-adf5-09b3fd26434d-config-volume\") pod \"dns-default-s9z7s\" (UID: \"a0fce8f4-676e-4fec-adf5-09b3fd26434d\") " pod="openshift-dns/dns-default-s9z7s"
Jul 13 19:22:18.621044 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.620791    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m87k\" (UniqueName: \"kubernetes.io/projected/a0fce8f4-676e-4fec-adf5-09b3fd26434d-kube-api-access-5m87k\") pod \"dns-default-s9z7s\" (UID: \"a0fce8f4-676e-4fec-adf5-09b3fd26434d\") " pod="openshift-dns/dns-default-s9z7s"
Jul 13 19:22:18.621044 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.620822    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51a39015-504a-4953-82e0-45eb6b25be9d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7556r\" (UID: \"51a39015-504a-4953-82e0-45eb6b25be9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7556r"
Jul 13 19:22:18.621044 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.620823    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fce914fd-6bde-48a0-9215-e66d18b67607-config\") pod \"service-ca-operator-7695fc6c8c-hvc57\" (UID: \"fce914fd-6bde-48a0-9215-e66d18b67607\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvc57"
Jul 13 19:22:18.621044 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.620879    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5945d50-bd96-4904-96ba-ea69aaef93d8-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-nnsb7\" (UID: \"c5945d50-bd96-4904-96ba-ea69aaef93d8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7"
Jul 13 19:22:18.621044 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.620882    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b1572934-6111-4394-9114-396122b025e7-registry-certificates\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.621044 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.620904    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0fce8f4-676e-4fec-adf5-09b3fd26434d-metrics-tls\") pod \"dns-default-s9z7s\" (UID: \"a0fce8f4-676e-4fec-adf5-09b3fd26434d\") " pod="openshift-dns/dns-default-s9z7s"
Jul 13 19:22:18.621044 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.620935    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-default-certificate\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:18.621044 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:18.620978    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 19:22:18.621044 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:18.621023    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0fce8f4-676e-4fec-adf5-09b3fd26434d-metrics-tls podName:a0fce8f4-676e-4fec-adf5-09b3fd26434d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:19.121011531 +0000 UTC m=+36.634888116 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a0fce8f4-676e-4fec-adf5-09b3fd26434d-metrics-tls") pod "dns-default-s9z7s" (UID: "a0fce8f4-676e-4fec-adf5-09b3fd26434d") : secret "dns-default-metrics-tls" not found
Jul 13 19:22:18.621044 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.620976    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jp7f\" (UniqueName: \"kubernetes.io/projected/507498fd-317d-48a0-b218-ea88bf7e6a58-kube-api-access-4jp7f\") pod \"volume-data-source-validator-78495597f6-mmdms\" (UID: \"507498fd-317d-48a0-b218-ea88bf7e6a58\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-mmdms"
Jul 13 19:22:18.621587 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.621061    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/319e53f6-7d9e-48f3-b9f6-001ada635739-trusted-ca\") pod \"console-operator-7866497494-c55st\" (UID: \"319e53f6-7d9e-48f3-b9f6-001ada635739\") " pod="openshift-console-operator/console-operator-7866497494-c55st"
Jul 13 19:22:18.621587 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.621093    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b1572934-6111-4394-9114-396122b025e7-ca-trust-extracted\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.621587 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.621117    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fce914fd-6bde-48a0-9215-e66d18b67607-serving-cert\") pod \"service-ca-operator-7695fc6c8c-hvc57\" (UID: \"fce914fd-6bde-48a0-9215-e66d18b67607\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvc57"
Jul 13 19:22:18.621587 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.621145    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm8n6\" (UniqueName: \"kubernetes.io/projected/95371626-7fce-48f7-a5ba-91cc14074e85-kube-api-access-pm8n6\") pod \"ingress-canary-59rng\" (UID: \"95371626-7fce-48f7-a5ba-91cc14074e85\") " pod="openshift-ingress-canary/ingress-canary-59rng"
Jul 13 19:22:18.621587 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.621184    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-bound-sa-token\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.621587 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.621209    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rcnz\" (UniqueName: \"kubernetes.io/projected/c889ca5e-ea7c-4e73-b692-6861504c2c8a-kube-api-access-5rcnz\") pod \"network-check-source-869dcfcdcd-gg4gl\" (UID: \"c889ca5e-ea7c-4e73-b692-6861504c2c8a\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-gg4gl"
Jul 13 19:22:18.621587 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.621242    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b1572934-6111-4394-9114-396122b025e7-installation-pull-secrets\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.621587 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.621261    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0fce8f4-676e-4fec-adf5-09b3fd26434d-tmp-dir\") pod \"dns-default-s9z7s\" (UID: \"a0fce8f4-676e-4fec-adf5-09b3fd26434d\") " pod="openshift-dns/dns-default-s9z7s"
Jul 13 19:22:18.621587 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.621551    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b1572934-6111-4394-9114-396122b025e7-trusted-ca\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.621587 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.621572    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b1572934-6111-4394-9114-396122b025e7-ca-trust-extracted\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.621587 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.621582    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a0fce8f4-676e-4fec-adf5-09b3fd26434d-config-volume\") pod \"dns-default-s9z7s\" (UID: \"a0fce8f4-676e-4fec-adf5-09b3fd26434d\") " pod="openshift-dns/dns-default-s9z7s"
Jul 13 19:22:18.622120 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.621891    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/319e53f6-7d9e-48f3-b9f6-001ada635739-trusted-ca\") pod \"console-operator-7866497494-c55st\" (UID: \"319e53f6-7d9e-48f3-b9f6-001ada635739\") " pod="openshift-console-operator/console-operator-7866497494-c55st"
Jul 13 19:22:18.625620 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.625526    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-p47mr\" (UID: \"f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-p47mr"
Jul 13 19:22:18.625620 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.625554    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fce914fd-6bde-48a0-9215-e66d18b67607-serving-cert\") pod \"service-ca-operator-7695fc6c8c-hvc57\" (UID: \"fce914fd-6bde-48a0-9215-e66d18b67607\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvc57"
Jul 13 19:22:18.625852 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.625666    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/319e53f6-7d9e-48f3-b9f6-001ada635739-serving-cert\") pod \"console-operator-7866497494-c55st\" (UID: \"319e53f6-7d9e-48f3-b9f6-001ada635739\") " pod="openshift-console-operator/console-operator-7866497494-c55st"
Jul 13 19:22:18.625852 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.625707    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b1572934-6111-4394-9114-396122b025e7-image-registry-private-configuration\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.626264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.626211    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b1572934-6111-4394-9114-396122b025e7-installation-pull-secrets\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.634621 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.634592    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x7s6\" (UniqueName: \"kubernetes.io/projected/fce914fd-6bde-48a0-9215-e66d18b67607-kube-api-access-6x7s6\") pod \"service-ca-operator-7695fc6c8c-hvc57\" (UID: \"fce914fd-6bde-48a0-9215-e66d18b67607\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvc57"
Jul 13 19:22:18.635226 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.635182    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jrbh\" (UniqueName: \"kubernetes.io/projected/f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc-kube-api-access-8jrbh\") pod \"kube-storage-version-migrator-operator-799cb7f958-p47mr\" (UID: \"f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-p47mr"
Jul 13 19:22:18.635782 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.635758    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jp7f\" (UniqueName: \"kubernetes.io/projected/507498fd-317d-48a0-b218-ea88bf7e6a58-kube-api-access-4jp7f\") pod \"volume-data-source-validator-78495597f6-mmdms\" (UID: \"507498fd-317d-48a0-b218-ea88bf7e6a58\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-mmdms"
Jul 13 19:22:18.636207 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.636138    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m87k\" (UniqueName: \"kubernetes.io/projected/a0fce8f4-676e-4fec-adf5-09b3fd26434d-kube-api-access-5m87k\") pod \"dns-default-s9z7s\" (UID: \"a0fce8f4-676e-4fec-adf5-09b3fd26434d\") " pod="openshift-dns/dns-default-s9z7s"
Jul 13 19:22:18.636794 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.636754    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbgdv\" (UniqueName: \"kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-kube-api-access-zbgdv\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.637096 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.637070    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-bound-sa-token\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:18.638674 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.638648    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c6m5\" (UniqueName: \"kubernetes.io/projected/319e53f6-7d9e-48f3-b9f6-001ada635739-kube-api-access-5c6m5\") pod \"console-operator-7866497494-c55st\" (UID: \"319e53f6-7d9e-48f3-b9f6-001ada635739\") " pod="openshift-console-operator/console-operator-7866497494-c55st"
Jul 13 19:22:18.713301 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.713268    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-c55st"
Jul 13 19:22:18.722607 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.722565    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c5945d50-bd96-4904-96ba-ea69aaef93d8-snapshots\") pod \"insights-operator-68d9dd94cf-nnsb7\" (UID: \"c5945d50-bd96-4904-96ba-ea69aaef93d8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7"
Jul 13 19:22:18.722781 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.722623    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c0acc219-19c5-4387-a6d3-7c83cffb00df-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-gr7gb\" (UID: \"c0acc219-19c5-4387-a6d3-7c83cffb00df\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb"
Jul 13 19:22:18.722781 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.722664    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddf2l\" (UniqueName: \"kubernetes.io/projected/c0acc219-19c5-4387-a6d3-7c83cffb00df-kube-api-access-ddf2l\") pod \"cluster-monitoring-operator-5f7f494dd4-gr7gb\" (UID: \"c0acc219-19c5-4387-a6d3-7c83cffb00df\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb"
Jul 13 19:22:18.722781 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.722694    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c5945d50-bd96-4904-96ba-ea69aaef93d8-serving-cert\") pod \"insights-operator-68d9dd94cf-nnsb7\" (UID: \"c5945d50-bd96-4904-96ba-ea69aaef93d8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7"
Jul 13 19:22:18.722781 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.722721    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0acc219-19c5-4387-a6d3-7c83cffb00df-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gr7gb\" (UID: \"c0acc219-19c5-4387-a6d3-7c83cffb00df\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb"
Jul 13 19:22:18.722781 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.722759    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5945d50-bd96-4904-96ba-ea69aaef93d8-tmp\") pod \"insights-operator-68d9dd94cf-nnsb7\" (UID: \"c5945d50-bd96-4904-96ba-ea69aaef93d8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7"
Jul 13 19:22:18.723043 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.722784    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfhsz\" (UniqueName: \"kubernetes.io/projected/c5945d50-bd96-4904-96ba-ea69aaef93d8-kube-api-access-zfhsz\") pod \"insights-operator-68d9dd94cf-nnsb7\" (UID: \"c5945d50-bd96-4904-96ba-ea69aaef93d8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7"
Jul 13 19:22:18.723043 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.722818    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnqss\" (UniqueName: \"kubernetes.io/projected/a55203b2-d840-4319-9ced-26799e123698-kube-api-access-rnqss\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:18.723043 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.722882    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-metrics-certs\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:18.723043 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.722905    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j447v\" (UniqueName: \"kubernetes.io/projected/51a39015-504a-4953-82e0-45eb6b25be9d-kube-api-access-j447v\") pod \"cluster-samples-operator-65db858894-7556r\" (UID: \"51a39015-504a-4953-82e0-45eb6b25be9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7556r"
Jul 13 19:22:18.723043 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.722934    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51a39015-504a-4953-82e0-45eb6b25be9d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7556r\" (UID: \"51a39015-504a-4953-82e0-45eb6b25be9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7556r"
Jul 13 19:22:18.723043 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.722965    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5945d50-bd96-4904-96ba-ea69aaef93d8-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-nnsb7\" (UID: \"c5945d50-bd96-4904-96ba-ea69aaef93d8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7"
Jul 13 19:22:18.723043 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.722989    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-default-certificate\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:18.723043 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.723018    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm8n6\" (UniqueName: \"kubernetes.io/projected/95371626-7fce-48f7-a5ba-91cc14074e85-kube-api-access-pm8n6\") pod \"ingress-canary-59rng\" (UID: \"95371626-7fce-48f7-a5ba-91cc14074e85\") " pod="openshift-ingress-canary/ingress-canary-59rng"
Jul 13 19:22:18.723043 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.723043    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rcnz\" (UniqueName: \"kubernetes.io/projected/c889ca5e-ea7c-4e73-b692-6861504c2c8a-kube-api-access-5rcnz\") pod \"network-check-source-869dcfcdcd-gg4gl\" (UID: \"c889ca5e-ea7c-4e73-b692-6861504c2c8a\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-gg4gl"
Jul 13 19:22:18.723441 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.723063    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95371626-7fce-48f7-a5ba-91cc14074e85-cert\") pod \"ingress-canary-59rng\" (UID: \"95371626-7fce-48f7-a5ba-91cc14074e85\") " pod="openshift-ingress-canary/ingress-canary-59rng"
Jul 13 19:22:18.723441 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.723089    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a55203b2-d840-4319-9ced-26799e123698-service-ca-bundle\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:18.723441 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.723111    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5945d50-bd96-4904-96ba-ea69aaef93d8-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-nnsb7\" (UID: \"c5945d50-bd96-4904-96ba-ea69aaef93d8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7"
Jul 13 19:22:18.723441 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.723136    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-stats-auth\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:18.723441 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.723280    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c5945d50-bd96-4904-96ba-ea69aaef93d8-snapshots\") pod \"insights-operator-68d9dd94cf-nnsb7\" (UID: \"c5945d50-bd96-4904-96ba-ea69aaef93d8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7"
Jul 13 19:22:18.723441 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:18.723331    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 19:22:18.723441 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:18.723400    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0acc219-19c5-4387-a6d3-7c83cffb00df-cluster-monitoring-operator-tls podName:c0acc219-19c5-4387-a6d3-7c83cffb00df nodeName:}" failed. No retries permitted until 2026-07-13 19:22:19.223381622 +0000 UTC m=+36.737258207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c0acc219-19c5-4387-a6d3-7c83cffb00df-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-gr7gb" (UID: "c0acc219-19c5-4387-a6d3-7c83cffb00df") : secret "cluster-monitoring-operator-tls" not found
Jul 13 19:22:18.723441 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.723407    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c0acc219-19c5-4387-a6d3-7c83cffb00df-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-gr7gb\" (UID: \"c0acc219-19c5-4387-a6d3-7c83cffb00df\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb"
Jul 13 19:22:18.723820 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:18.723466    2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 13 19:22:18.723820 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:18.723489    2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 19:22:18.723820 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:18.723537    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-metrics-certs podName:a55203b2-d840-4319-9ced-26799e123698 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:19.223502094 +0000 UTC m=+36.737378678 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-metrics-certs") pod "router-default-68dc6f48c8-qx2hp" (UID: "a55203b2-d840-4319-9ced-26799e123698") : secret "router-metrics-certs-default" not found
Jul 13 19:22:18.723820 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:18.723557    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51a39015-504a-4953-82e0-45eb6b25be9d-samples-operator-tls podName:51a39015-504a-4953-82e0-45eb6b25be9d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:19.223546933 +0000 UTC m=+36.737423524 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/51a39015-504a-4953-82e0-45eb6b25be9d-samples-operator-tls") pod "cluster-samples-operator-65db858894-7556r" (UID: "51a39015-504a-4953-82e0-45eb6b25be9d") : secret "samples-operator-tls" not found
Jul 13 19:22:18.723820 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.723749    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5945d50-bd96-4904-96ba-ea69aaef93d8-tmp\") pod \"insights-operator-68d9dd94cf-nnsb7\" (UID: \"c5945d50-bd96-4904-96ba-ea69aaef93d8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7"
Jul 13 19:22:18.726469 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:18.723860    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a55203b2-d840-4319-9ced-26799e123698-service-ca-bundle podName:a55203b2-d840-4319-9ced-26799e123698 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:19.223845484 +0000 UTC m=+36.737722071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a55203b2-d840-4319-9ced-26799e123698-service-ca-bundle") pod "router-default-68dc6f48c8-qx2hp" (UID: "a55203b2-d840-4319-9ced-26799e123698") : configmap references non-existent config key: service-ca.crt
Jul 13 19:22:18.726469 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.724005    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5945d50-bd96-4904-96ba-ea69aaef93d8-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-nnsb7\" (UID: \"c5945d50-bd96-4904-96ba-ea69aaef93d8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7"
Jul 13 19:22:18.726469 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.724589    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5945d50-bd96-4904-96ba-ea69aaef93d8-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-nnsb7\" (UID: \"c5945d50-bd96-4904-96ba-ea69aaef93d8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7"
Jul 13 19:22:18.726469 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.726382    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-default-certificate\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:18.726469 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.726389    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-stats-auth\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:18.726469 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.726409    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95371626-7fce-48f7-a5ba-91cc14074e85-cert\") pod \"ingress-canary-59rng\" (UID: \"95371626-7fce-48f7-a5ba-91cc14074e85\") " pod="openshift-ingress-canary/ingress-canary-59rng"
Jul 13 19:22:18.726469 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.726439    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c5945d50-bd96-4904-96ba-ea69aaef93d8-serving-cert\") pod \"insights-operator-68d9dd94cf-nnsb7\" (UID: \"c5945d50-bd96-4904-96ba-ea69aaef93d8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7"
Jul 13 19:22:18.731770 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.731746    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-mmdms"
Jul 13 19:22:18.732243 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.732215    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm8n6\" (UniqueName: \"kubernetes.io/projected/95371626-7fce-48f7-a5ba-91cc14074e85-kube-api-access-pm8n6\") pod \"ingress-canary-59rng\" (UID: \"95371626-7fce-48f7-a5ba-91cc14074e85\") " pod="openshift-ingress-canary/ingress-canary-59rng"
Jul 13 19:22:18.732499 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.732453    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfhsz\" (UniqueName: \"kubernetes.io/projected/c5945d50-bd96-4904-96ba-ea69aaef93d8-kube-api-access-zfhsz\") pod \"insights-operator-68d9dd94cf-nnsb7\" (UID: \"c5945d50-bd96-4904-96ba-ea69aaef93d8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7"
Jul 13 19:22:18.732819 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.732610    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnqss\" (UniqueName: \"kubernetes.io/projected/a55203b2-d840-4319-9ced-26799e123698-kube-api-access-rnqss\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:18.733413 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.733361    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddf2l\" (UniqueName: \"kubernetes.io/projected/c0acc219-19c5-4387-a6d3-7c83cffb00df-kube-api-access-ddf2l\") pod \"cluster-monitoring-operator-5f7f494dd4-gr7gb\" (UID: \"c0acc219-19c5-4387-a6d3-7c83cffb00df\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb"
Jul 13 19:22:18.733829 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.733807    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j447v\" (UniqueName: \"kubernetes.io/projected/51a39015-504a-4953-82e0-45eb6b25be9d-kube-api-access-j447v\") pod \"cluster-samples-operator-65db858894-7556r\" (UID: \"51a39015-504a-4953-82e0-45eb6b25be9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7556r"
Jul 13 19:22:18.734927 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.734900    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rcnz\" (UniqueName: \"kubernetes.io/projected/c889ca5e-ea7c-4e73-b692-6861504c2c8a-kube-api-access-5rcnz\") pod \"network-check-source-869dcfcdcd-gg4gl\" (UID: \"c889ca5e-ea7c-4e73-b692-6861504c2c8a\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-gg4gl"
Jul 13 19:22:18.738399 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.738380    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvc57"
Jul 13 19:22:18.747377 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.747355    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-p47mr"
Jul 13 19:22:18.776371 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.776179    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7"
Jul 13 19:22:18.786457 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.786418    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-59rng"
Jul 13 19:22:18.793260 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:18.793236    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-gg4gl"
Jul 13 19:22:19.126057 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:19.126019    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:19.126510 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:19.126104    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0fce8f4-676e-4fec-adf5-09b3fd26434d-metrics-tls\") pod \"dns-default-s9z7s\" (UID: \"a0fce8f4-676e-4fec-adf5-09b3fd26434d\") " pod="openshift-dns/dns-default-s9z7s"
Jul 13 19:22:19.126510 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:19.126206    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 19:22:19.126510 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:19.126216    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54bb1498-3f40-47ca-a33c-eaebcf72090b-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-dqtq2\" (UID: \"54bb1498-3f40-47ca-a33c-eaebcf72090b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-dqtq2"
Jul 13 19:22:19.126510 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:19.126227    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b77547c98-zzq67: secret "image-registry-tls" not found
Jul 13 19:22:19.126510 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:19.126255    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 19:22:19.126510 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:19.126349    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls podName:b1572934-6111-4394-9114-396122b025e7 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:20.126329035 +0000 UTC m=+37.640205636 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls") pod "image-registry-5b77547c98-zzq67" (UID: "b1572934-6111-4394-9114-396122b025e7") : secret "image-registry-tls" not found
Jul 13 19:22:19.126510 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:19.126406    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0fce8f4-676e-4fec-adf5-09b3fd26434d-metrics-tls podName:a0fce8f4-676e-4fec-adf5-09b3fd26434d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:20.126389529 +0000 UTC m=+37.640266116 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a0fce8f4-676e-4fec-adf5-09b3fd26434d-metrics-tls") pod "dns-default-s9z7s" (UID: "a0fce8f4-676e-4fec-adf5-09b3fd26434d") : secret "dns-default-metrics-tls" not found
Jul 13 19:22:19.126510 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:19.126277    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 13 19:22:19.126510 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:19.126469    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54bb1498-3f40-47ca-a33c-eaebcf72090b-networking-console-plugin-cert podName:54bb1498-3f40-47ca-a33c-eaebcf72090b nodeName:}" failed. No retries permitted until 2026-07-13 19:22:20.12645711 +0000 UTC m=+37.640333697 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/54bb1498-3f40-47ca-a33c-eaebcf72090b-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-dqtq2" (UID: "54bb1498-3f40-47ca-a33c-eaebcf72090b") : secret "networking-console-plugin-cert" not found
Jul 13 19:22:19.226842 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:19.226807    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a55203b2-d840-4319-9ced-26799e123698-service-ca-bundle\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:19.227033 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:19.226897    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0acc219-19c5-4387-a6d3-7c83cffb00df-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gr7gb\" (UID: \"c0acc219-19c5-4387-a6d3-7c83cffb00df\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb"
Jul 13 19:22:19.227033 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:19.226948    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a55203b2-d840-4319-9ced-26799e123698-service-ca-bundle podName:a55203b2-d840-4319-9ced-26799e123698 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:20.226928268 +0000 UTC m=+37.740804851 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a55203b2-d840-4319-9ced-26799e123698-service-ca-bundle") pod "router-default-68dc6f48c8-qx2hp" (UID: "a55203b2-d840-4319-9ced-26799e123698") : configmap references non-existent config key: service-ca.crt
Jul 13 19:22:19.227033 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:19.227011    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-metrics-certs\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:19.227218 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:19.227048    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51a39015-504a-4953-82e0-45eb6b25be9d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7556r\" (UID: \"51a39015-504a-4953-82e0-45eb6b25be9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7556r"
Jul 13 19:22:19.227218 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:19.227016    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 19:22:19.227218 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:19.227125    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0acc219-19c5-4387-a6d3-7c83cffb00df-cluster-monitoring-operator-tls podName:c0acc219-19c5-4387-a6d3-7c83cffb00df nodeName:}" failed. No retries permitted until 2026-07-13 19:22:20.227110862 +0000 UTC m=+37.740987462 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c0acc219-19c5-4387-a6d3-7c83cffb00df-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-gr7gb" (UID: "c0acc219-19c5-4387-a6d3-7c83cffb00df") : secret "cluster-monitoring-operator-tls" not found
Jul 13 19:22:19.227218 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:19.227072    2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 13 19:22:19.227218 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:19.227173    2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 19:22:19.227218 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:19.227189    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-metrics-certs podName:a55203b2-d840-4319-9ced-26799e123698 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:20.227180223 +0000 UTC m=+37.741056812 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-metrics-certs") pod "router-default-68dc6f48c8-qx2hp" (UID: "a55203b2-d840-4319-9ced-26799e123698") : secret "router-metrics-certs-default" not found
Jul 13 19:22:19.227559 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:19.227224    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51a39015-504a-4953-82e0-45eb6b25be9d-samples-operator-tls podName:51a39015-504a-4953-82e0-45eb6b25be9d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:20.22721002 +0000 UTC m=+37.741086606 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/51a39015-504a-4953-82e0-45eb6b25be9d-samples-operator-tls") pod "cluster-samples-operator-65db858894-7556r" (UID: "51a39015-504a-4953-82e0-45eb6b25be9d") : secret "samples-operator-tls" not found
Jul 13 19:22:19.751064 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:19.751027    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-mmdms"]
Jul 13 19:22:19.756162 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:19.755560    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-c55st"]
Jul 13 19:22:19.757030 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:19.757001    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvc57"]
Jul 13 19:22:19.758316 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:19.758292    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-p47mr"]
Jul 13 19:22:19.769296 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:19.769276    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-59rng"]
Jul 13 19:22:19.775465 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:19.775450    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-nnsb7"]
Jul 13 19:22:19.781930 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:19.781905    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-gg4gl"]
Jul 13 19:22:19.791585 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:22:19.791539    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod507498fd_317d_48a0_b218_ea88bf7e6a58.slice/crio-29fdff57f36fff3da979f50b60a64c7b71ccd5759c7dbebd23b0f11530882800 WatchSource:0}: Error finding container 29fdff57f36fff3da979f50b60a64c7b71ccd5759c7dbebd23b0f11530882800: Status 404 returned error can't find the container with id 29fdff57f36fff3da979f50b60a64c7b71ccd5759c7dbebd23b0f11530882800
Jul 13 19:22:19.792547 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:22:19.792503    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod319e53f6_7d9e_48f3_b9f6_001ada635739.slice/crio-8459f8327f55c133a0335b87285a5500c055974380d85f72b942b4c2f9102b0d WatchSource:0}: Error finding container 8459f8327f55c133a0335b87285a5500c055974380d85f72b942b4c2f9102b0d: Status 404 returned error can't find the container with id 8459f8327f55c133a0335b87285a5500c055974380d85f72b942b4c2f9102b0d
Jul 13 19:22:19.795822 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:22:19.793404    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfce914fd_6bde_48a0_9215_e66d18b67607.slice/crio-90fb21973bba656f1865e647d46a43c454584e6b9d6f8cac6d351050285f81f8 WatchSource:0}: Error finding container 90fb21973bba656f1865e647d46a43c454584e6b9d6f8cac6d351050285f81f8: Status 404 returned error can't find the container with id 90fb21973bba656f1865e647d46a43c454584e6b9d6f8cac6d351050285f81f8
Jul 13 19:22:19.795822 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:22:19.794671    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6c9fcfe_e75f_4c3d_bbf0_baee21960fbc.slice/crio-c66ee6d6401de9ce97212c2935d8e4bd100103f0fe13e0af7bf43d8a5e82a2c5 WatchSource:0}: Error finding container c66ee6d6401de9ce97212c2935d8e4bd100103f0fe13e0af7bf43d8a5e82a2c5: Status 404 returned error can't find the container with id c66ee6d6401de9ce97212c2935d8e4bd100103f0fe13e0af7bf43d8a5e82a2c5
Jul 13 19:22:20.042811 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.042603    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:22:20.042975 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.042607    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:22:20.042975 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.042610    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:22:20.045516 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.045479    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 13 19:22:20.045713 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.045521    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 13 19:22:20.045713 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.045526    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ztp4q\""
Jul 13 19:22:20.045713 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.045551    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fp668\""
Jul 13 19:22:20.137198 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.137165    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0fce8f4-676e-4fec-adf5-09b3fd26434d-metrics-tls\") pod \"dns-default-s9z7s\" (UID: \"a0fce8f4-676e-4fec-adf5-09b3fd26434d\") " pod="openshift-dns/dns-default-s9z7s"
Jul 13 19:22:20.137590 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.137239    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54bb1498-3f40-47ca-a33c-eaebcf72090b-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-dqtq2\" (UID: \"54bb1498-3f40-47ca-a33c-eaebcf72090b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-dqtq2"
Jul 13 19:22:20.137590 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.137289    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:20.137590 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:20.137346    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 19:22:20.137590 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:20.137376    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 19:22:20.137590 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:20.137383    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 13 19:22:20.137590 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:20.137421    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0fce8f4-676e-4fec-adf5-09b3fd26434d-metrics-tls podName:a0fce8f4-676e-4fec-adf5-09b3fd26434d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:22.137404373 +0000 UTC m=+39.651280960 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a0fce8f4-676e-4fec-adf5-09b3fd26434d-metrics-tls") pod "dns-default-s9z7s" (UID: "a0fce8f4-676e-4fec-adf5-09b3fd26434d") : secret "dns-default-metrics-tls" not found
Jul 13 19:22:20.137590 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:20.137449    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54bb1498-3f40-47ca-a33c-eaebcf72090b-networking-console-plugin-cert podName:54bb1498-3f40-47ca-a33c-eaebcf72090b nodeName:}" failed. No retries permitted until 2026-07-13 19:22:22.137433449 +0000 UTC m=+39.651310059 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/54bb1498-3f40-47ca-a33c-eaebcf72090b-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-dqtq2" (UID: "54bb1498-3f40-47ca-a33c-eaebcf72090b") : secret "networking-console-plugin-cert" not found
Jul 13 19:22:20.137590 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:20.137390    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b77547c98-zzq67: secret "image-registry-tls" not found
Jul 13 19:22:20.137590 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:20.137492    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls podName:b1572934-6111-4394-9114-396122b025e7 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:22.137483297 +0000 UTC m=+39.651359883 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls") pod "image-registry-5b77547c98-zzq67" (UID: "b1572934-6111-4394-9114-396122b025e7") : secret "image-registry-tls" not found
Jul 13 19:22:20.199100 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.199068    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-59rng" event={"ID":"95371626-7fce-48f7-a5ba-91cc14074e85","Type":"ContainerStarted","Data":"9a96d126e7e6dd1f0616da3a6eb97df194571e4cab30194a9dfeb6fdd6094d3c"}
Jul 13 19:22:20.200061 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.200034    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvc57" event={"ID":"fce914fd-6bde-48a0-9215-e66d18b67607","Type":"ContainerStarted","Data":"90fb21973bba656f1865e647d46a43c454584e6b9d6f8cac6d351050285f81f8"}
Jul 13 19:22:20.201055 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.201029    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-mmdms" event={"ID":"507498fd-317d-48a0-b218-ea88bf7e6a58","Type":"ContainerStarted","Data":"29fdff57f36fff3da979f50b60a64c7b71ccd5759c7dbebd23b0f11530882800"}
Jul 13 19:22:20.202123 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.202105    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-c55st" event={"ID":"319e53f6-7d9e-48f3-b9f6-001ada635739","Type":"ContainerStarted","Data":"8459f8327f55c133a0335b87285a5500c055974380d85f72b942b4c2f9102b0d"}
Jul 13 19:22:20.204651 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.204615    2573 generic.go:356] "Generic (PLEG): container finished" podID="86a1f531-5f82-4b77-b776-762ae3ff6170" containerID="a0038cc41c81fa006555e8285831d6e2311483e324a451d58ac17e0db472afb6" exitCode=0
Jul 13 19:22:20.204760 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.204669    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wsf2z" event={"ID":"86a1f531-5f82-4b77-b776-762ae3ff6170","Type":"ContainerDied","Data":"a0038cc41c81fa006555e8285831d6e2311483e324a451d58ac17e0db472afb6"}
Jul 13 19:22:20.205833 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.205814    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-gg4gl" event={"ID":"c889ca5e-ea7c-4e73-b692-6861504c2c8a","Type":"ContainerStarted","Data":"a5585c5f1c06fc3b6292f044356ce7884d4081a041a729d08bebe55bc0b2ae82"}
Jul 13 19:22:20.206923 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.206903    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7" event={"ID":"c5945d50-bd96-4904-96ba-ea69aaef93d8","Type":"ContainerStarted","Data":"1fc8a59a60c878e6bed54da0dc94365058a4961d2557deda8ba0c95f64b34d63"}
Jul 13 19:22:20.207942 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.207919    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-p47mr" event={"ID":"f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc","Type":"ContainerStarted","Data":"c66ee6d6401de9ce97212c2935d8e4bd100103f0fe13e0af7bf43d8a5e82a2c5"}
Jul 13 19:22:20.238471 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.238444    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a55203b2-d840-4319-9ced-26799e123698-service-ca-bundle\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:20.238578 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.238513    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0acc219-19c5-4387-a6d3-7c83cffb00df-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gr7gb\" (UID: \"c0acc219-19c5-4387-a6d3-7c83cffb00df\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb"
Jul 13 19:22:20.238578 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.238572    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-metrics-certs\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:20.238711 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:20.238605    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a55203b2-d840-4319-9ced-26799e123698-service-ca-bundle podName:a55203b2-d840-4319-9ced-26799e123698 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:22.238587379 +0000 UTC m=+39.752463963 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a55203b2-d840-4319-9ced-26799e123698-service-ca-bundle") pod "router-default-68dc6f48c8-qx2hp" (UID: "a55203b2-d840-4319-9ced-26799e123698") : configmap references non-existent config key: service-ca.crt
Jul 13 19:22:20.238711 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:20.238653    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51a39015-504a-4953-82e0-45eb6b25be9d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7556r\" (UID: \"51a39015-504a-4953-82e0-45eb6b25be9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7556r"
Jul 13 19:22:20.238711 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:20.238698    2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 13 19:22:20.238845 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:20.238698    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 19:22:20.238845 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:20.238753    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-metrics-certs podName:a55203b2-d840-4319-9ced-26799e123698 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:22.238737659 +0000 UTC m=+39.752614270 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-metrics-certs") pod "router-default-68dc6f48c8-qx2hp" (UID: "a55203b2-d840-4319-9ced-26799e123698") : secret "router-metrics-certs-default" not found
Jul 13 19:22:20.238845 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:20.238777    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0acc219-19c5-4387-a6d3-7c83cffb00df-cluster-monitoring-operator-tls podName:c0acc219-19c5-4387-a6d3-7c83cffb00df nodeName:}" failed. No retries permitted until 2026-07-13 19:22:22.238764436 +0000 UTC m=+39.752641026 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c0acc219-19c5-4387-a6d3-7c83cffb00df-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-gr7gb" (UID: "c0acc219-19c5-4387-a6d3-7c83cffb00df") : secret "cluster-monitoring-operator-tls" not found
Jul 13 19:22:20.238845 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:20.238779    2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 19:22:20.238845 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:20.238824    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51a39015-504a-4953-82e0-45eb6b25be9d-samples-operator-tls podName:51a39015-504a-4953-82e0-45eb6b25be9d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:22.23881589 +0000 UTC m=+39.752692479 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/51a39015-504a-4953-82e0-45eb6b25be9d-samples-operator-tls") pod "cluster-samples-operator-65db858894-7556r" (UID: "51a39015-504a-4953-82e0-45eb6b25be9d") : secret "samples-operator-tls" not found
Jul 13 19:22:21.221140 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:21.221094    2573 generic.go:356] "Generic (PLEG): container finished" podID="86a1f531-5f82-4b77-b776-762ae3ff6170" containerID="8a8449cdc5594aaaf69f798fec8799faa596b79713444f3ecf62111f56c77117" exitCode=0
Jul 13 19:22:21.221720 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:21.221172    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wsf2z" event={"ID":"86a1f531-5f82-4b77-b776-762ae3ff6170","Type":"ContainerDied","Data":"8a8449cdc5594aaaf69f798fec8799faa596b79713444f3ecf62111f56c77117"}
Jul 13 19:22:22.158500 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:22.158391    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:22.158500 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:22.158498    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0fce8f4-676e-4fec-adf5-09b3fd26434d-metrics-tls\") pod \"dns-default-s9z7s\" (UID: \"a0fce8f4-676e-4fec-adf5-09b3fd26434d\") " pod="openshift-dns/dns-default-s9z7s"
Jul 13 19:22:22.158735 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:22.158597    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54bb1498-3f40-47ca-a33c-eaebcf72090b-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-dqtq2\" (UID: \"54bb1498-3f40-47ca-a33c-eaebcf72090b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-dqtq2"
Jul 13 19:22:22.158801 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:22.158776    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 13 19:22:22.158855 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:22.158849    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54bb1498-3f40-47ca-a33c-eaebcf72090b-networking-console-plugin-cert podName:54bb1498-3f40-47ca-a33c-eaebcf72090b nodeName:}" failed. No retries permitted until 2026-07-13 19:22:26.158829955 +0000 UTC m=+43.672706544 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/54bb1498-3f40-47ca-a33c-eaebcf72090b-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-dqtq2" (UID: "54bb1498-3f40-47ca-a33c-eaebcf72090b") : secret "networking-console-plugin-cert" not found
Jul 13 19:22:22.159286 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:22.159263    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 19:22:22.159286 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:22.159288    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b77547c98-zzq67: secret "image-registry-tls" not found
Jul 13 19:22:22.159432 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:22.159333    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls podName:b1572934-6111-4394-9114-396122b025e7 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:26.159317899 +0000 UTC m=+43.673194488 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls") pod "image-registry-5b77547c98-zzq67" (UID: "b1572934-6111-4394-9114-396122b025e7") : secret "image-registry-tls" not found
Jul 13 19:22:22.159432 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:22.159391    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 19:22:22.159432 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:22.159420    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0fce8f4-676e-4fec-adf5-09b3fd26434d-metrics-tls podName:a0fce8f4-676e-4fec-adf5-09b3fd26434d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:26.159411131 +0000 UTC m=+43.673287715 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a0fce8f4-676e-4fec-adf5-09b3fd26434d-metrics-tls") pod "dns-default-s9z7s" (UID: "a0fce8f4-676e-4fec-adf5-09b3fd26434d") : secret "dns-default-metrics-tls" not found
Jul 13 19:22:22.230328 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:22.230281    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wsf2z" event={"ID":"86a1f531-5f82-4b77-b776-762ae3ff6170","Type":"ContainerStarted","Data":"5e7f2d3dee0e628afbd36cf2824476dc86cbb9422748b4d0c72c59088cb9b304"}
Jul 13 19:22:22.256242 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:22.256083    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wsf2z" podStartSLOduration=8.876863286 podStartE2EDuration="39.256060348s" totalImagesPullingTime="30.379197062s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:43 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:22.255951594 +0000 UTC m=+39.769828240" watchObservedRunningTime="2026-07-13 19:22:22.256060348 +0000 UTC m=+39.769936965"
Jul 13 19:22:22.259589 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:22.259456    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a55203b2-d840-4319-9ced-26799e123698-service-ca-bundle\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:22.259589 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:22.259535    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0acc219-19c5-4387-a6d3-7c83cffb00df-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gr7gb\" (UID: \"c0acc219-19c5-4387-a6d3-7c83cffb00df\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb"
Jul 13 19:22:22.259803 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:22.259598    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-metrics-certs\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:22.259803 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:22.259650    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51a39015-504a-4953-82e0-45eb6b25be9d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7556r\" (UID: \"51a39015-504a-4953-82e0-45eb6b25be9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7556r"
Jul 13 19:22:22.259803 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:22.259694    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 19:22:22.259803 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:22.259699    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a55203b2-d840-4319-9ced-26799e123698-service-ca-bundle podName:a55203b2-d840-4319-9ced-26799e123698 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:26.259681457 +0000 UTC m=+43.773558046 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a55203b2-d840-4319-9ced-26799e123698-service-ca-bundle") pod "router-default-68dc6f48c8-qx2hp" (UID: "a55203b2-d840-4319-9ced-26799e123698") : configmap references non-existent config key: service-ca.crt
Jul 13 19:22:22.259803 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:22.259758    2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 19:22:22.259803 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:22.259808    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51a39015-504a-4953-82e0-45eb6b25be9d-samples-operator-tls podName:51a39015-504a-4953-82e0-45eb6b25be9d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:26.259793304 +0000 UTC m=+43.773669894 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/51a39015-504a-4953-82e0-45eb6b25be9d-samples-operator-tls") pod "cluster-samples-operator-65db858894-7556r" (UID: "51a39015-504a-4953-82e0-45eb6b25be9d") : secret "samples-operator-tls" not found
Jul 13 19:22:22.260113 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:22.259826    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0acc219-19c5-4387-a6d3-7c83cffb00df-cluster-monitoring-operator-tls podName:c0acc219-19c5-4387-a6d3-7c83cffb00df nodeName:}" failed. No retries permitted until 2026-07-13 19:22:26.259815791 +0000 UTC m=+43.773692379 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c0acc219-19c5-4387-a6d3-7c83cffb00df-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-gr7gb" (UID: "c0acc219-19c5-4387-a6d3-7c83cffb00df") : secret "cluster-monitoring-operator-tls" not found
Jul 13 19:22:22.260113 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:22.259894    2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 13 19:22:22.260113 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:22.259932    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-metrics-certs podName:a55203b2-d840-4319-9ced-26799e123698 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:26.259918946 +0000 UTC m=+43.773795539 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-metrics-certs") pod "router-default-68dc6f48c8-qx2hp" (UID: "a55203b2-d840-4319-9ced-26799e123698") : secret "router-metrics-certs-default" not found
Jul 13 19:22:26.198509 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:26.198464    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:26.199251 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:26.198679    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 19:22:26.199251 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:26.198691    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0fce8f4-676e-4fec-adf5-09b3fd26434d-metrics-tls\") pod \"dns-default-s9z7s\" (UID: \"a0fce8f4-676e-4fec-adf5-09b3fd26434d\") " pod="openshift-dns/dns-default-s9z7s"
Jul 13 19:22:26.199251 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:26.198807    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54bb1498-3f40-47ca-a33c-eaebcf72090b-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-dqtq2\" (UID: \"54bb1498-3f40-47ca-a33c-eaebcf72090b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-dqtq2"
Jul 13 19:22:26.199251 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:26.198704    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b77547c98-zzq67: secret "image-registry-tls" not found
Jul 13 19:22:26.199251 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:26.198887    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls podName:b1572934-6111-4394-9114-396122b025e7 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:34.198869293 +0000 UTC m=+51.712745880 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls") pod "image-registry-5b77547c98-zzq67" (UID: "b1572934-6111-4394-9114-396122b025e7") : secret "image-registry-tls" not found
Jul 13 19:22:26.199251 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:26.198758    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 19:22:26.199251 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:26.198926    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 13 19:22:26.199251 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:26.198967    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0fce8f4-676e-4fec-adf5-09b3fd26434d-metrics-tls podName:a0fce8f4-676e-4fec-adf5-09b3fd26434d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:34.198944471 +0000 UTC m=+51.712821078 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a0fce8f4-676e-4fec-adf5-09b3fd26434d-metrics-tls") pod "dns-default-s9z7s" (UID: "a0fce8f4-676e-4fec-adf5-09b3fd26434d") : secret "dns-default-metrics-tls" not found
Jul 13 19:22:26.199251 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:26.198989    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54bb1498-3f40-47ca-a33c-eaebcf72090b-networking-console-plugin-cert podName:54bb1498-3f40-47ca-a33c-eaebcf72090b nodeName:}" failed. No retries permitted until 2026-07-13 19:22:34.198978915 +0000 UTC m=+51.712855503 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/54bb1498-3f40-47ca-a33c-eaebcf72090b-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-dqtq2" (UID: "54bb1498-3f40-47ca-a33c-eaebcf72090b") : secret "networking-console-plugin-cert" not found
Jul 13 19:22:26.299783 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:26.299708    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0acc219-19c5-4387-a6d3-7c83cffb00df-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gr7gb\" (UID: \"c0acc219-19c5-4387-a6d3-7c83cffb00df\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb"
Jul 13 19:22:26.299973 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:26.299899    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 19:22:26.300058 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:26.299978    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0acc219-19c5-4387-a6d3-7c83cffb00df-cluster-monitoring-operator-tls podName:c0acc219-19c5-4387-a6d3-7c83cffb00df nodeName:}" failed. No retries permitted until 2026-07-13 19:22:34.299955083 +0000 UTC m=+51.813831667 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c0acc219-19c5-4387-a6d3-7c83cffb00df-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-gr7gb" (UID: "c0acc219-19c5-4387-a6d3-7c83cffb00df") : secret "cluster-monitoring-operator-tls" not found
Jul 13 19:22:26.300058 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:26.299898    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-metrics-certs\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:26.300058 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:26.299987    2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 13 19:22:26.300058 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:26.300036    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51a39015-504a-4953-82e0-45eb6b25be9d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7556r\" (UID: \"51a39015-504a-4953-82e0-45eb6b25be9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7556r"
Jul 13 19:22:26.300212 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:26.300079    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-metrics-certs podName:a55203b2-d840-4319-9ced-26799e123698 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:34.300062134 +0000 UTC m=+51.813938743 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-metrics-certs") pod "router-default-68dc6f48c8-qx2hp" (UID: "a55203b2-d840-4319-9ced-26799e123698") : secret "router-metrics-certs-default" not found
Jul 13 19:22:26.300212 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:26.300130    2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 19:22:26.300212 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:26.300163    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a55203b2-d840-4319-9ced-26799e123698-service-ca-bundle\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:26.300331 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:26.300215    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a55203b2-d840-4319-9ced-26799e123698-service-ca-bundle podName:a55203b2-d840-4319-9ced-26799e123698 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:34.300203545 +0000 UTC m=+51.814080138 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a55203b2-d840-4319-9ced-26799e123698-service-ca-bundle") pod "router-default-68dc6f48c8-qx2hp" (UID: "a55203b2-d840-4319-9ced-26799e123698") : configmap references non-existent config key: service-ca.crt
Jul 13 19:22:26.300331 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:26.300234    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51a39015-504a-4953-82e0-45eb6b25be9d-samples-operator-tls podName:51a39015-504a-4953-82e0-45eb6b25be9d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:34.300223388 +0000 UTC m=+51.814099972 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/51a39015-504a-4953-82e0-45eb6b25be9d-samples-operator-tls") pod "cluster-samples-operator-65db858894-7556r" (UID: "51a39015-504a-4953-82e0-45eb6b25be9d") : secret "samples-operator-tls" not found
Jul 13 19:22:28.249554 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.249245    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-gg4gl" event={"ID":"c889ca5e-ea7c-4e73-b692-6861504c2c8a","Type":"ContainerStarted","Data":"9f11f6733cfb10ad6e3178cc1bc11e644254d71b032c848feab13a4444ba1910"}
Jul 13 19:22:28.250869 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.250818    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7" event={"ID":"c5945d50-bd96-4904-96ba-ea69aaef93d8","Type":"ContainerStarted","Data":"0504b3b32425e8056096a3b86123f992f42c40b13a9206ae40153b78acc6c585"}
Jul 13 19:22:28.253106 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.253077    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-p47mr" event={"ID":"f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc","Type":"ContainerStarted","Data":"40b6ea5faa011b30f77971afa7af7ea9a5d2356f7b9cd0656fb85840e37a50a3"}
Jul 13 19:22:28.254938 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.254911    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-59rng" event={"ID":"95371626-7fce-48f7-a5ba-91cc14074e85","Type":"ContainerStarted","Data":"84beb15d866f43c1b87f20f6697d761f4c875e98e2199dffcb0a4bae55a46fc3"}
Jul 13 19:22:28.256624 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.256598    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvc57" event={"ID":"fce914fd-6bde-48a0-9215-e66d18b67607","Type":"ContainerStarted","Data":"c9a6ba0cbd9134b5b30572f0a6840159f70128be2568e1e312377c055b553086"}
Jul 13 19:22:28.258297 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.258271    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-mmdms" event={"ID":"507498fd-317d-48a0-b218-ea88bf7e6a58","Type":"ContainerStarted","Data":"896b934978c8f585a4df0c2c80e6c83c714e22cc2426b9f890a61c2104aa08ac"}
Jul 13 19:22:28.260138 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.260118    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/0.log"
Jul 13 19:22:28.260250 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.260160    2573 generic.go:356] "Generic (PLEG): container finished" podID="319e53f6-7d9e-48f3-b9f6-001ada635739" containerID="cfbd9f3244800c0df2adc5e76a210e42022c068db14ef5c3ead14652936b80b5" exitCode=255
Jul 13 19:22:28.260250 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.260200    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-c55st" event={"ID":"319e53f6-7d9e-48f3-b9f6-001ada635739","Type":"ContainerDied","Data":"cfbd9f3244800c0df2adc5e76a210e42022c068db14ef5c3ead14652936b80b5"}
Jul 13 19:22:28.260443 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.260426    2573 scope.go:117] "RemoveContainer" containerID="cfbd9f3244800c0df2adc5e76a210e42022c068db14ef5c3ead14652936b80b5"
Jul 13 19:22:28.266878 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.266813    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-gg4gl" podStartSLOduration=28.698163855 podStartE2EDuration="36.266792293s" totalImagesPullingTime="7.568628438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:21:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:28.265336393 +0000 UTC m=+45.779213025" watchObservedRunningTime="2026-07-13 19:22:28.266792293 +0000 UTC m=+45.780668940"
Jul 13 19:22:28.288526 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.288466    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7" podStartSLOduration=29.719802764 podStartE2EDuration="37.288425576s" totalImagesPullingTime="7.568622812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:21:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:28.287051692 +0000 UTC m=+45.800928299" watchObservedRunningTime="2026-07-13 19:22:28.288425576 +0000 UTC m=+45.802302182"
Jul 13 19:22:28.303783 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.303730    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvc57" podStartSLOduration=29.413584401 podStartE2EDuration="37.303707264s" totalImagesPullingTime="7.890122863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:21:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:28.303050961 +0000 UTC m=+45.816927586" watchObservedRunningTime="2026-07-13 19:22:28.303707264 +0000 UTC m=+45.817583849"
Jul 13 19:22:28.325140 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.325086    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-59rng" podStartSLOduration=2.747934249 podStartE2EDuration="10.325043996s" totalImagesPullingTime="7.577109747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:28.322020401 +0000 UTC m=+45.835897031" watchObservedRunningTime="2026-07-13 19:22:28.325043996 +0000 UTC m=+45.838920603"
Jul 13 19:22:28.340518 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.340340    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-mmdms" podStartSLOduration=29.78871221 podStartE2EDuration="37.34031985s" totalImagesPullingTime="7.55160764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:21:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:28.338885181 +0000 UTC m=+45.852761788" watchObservedRunningTime="2026-07-13 19:22:28.34031985 +0000 UTC m=+45.854196438"
Jul 13 19:22:28.714408 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.714370    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-c55st"
Jul 13 19:22:28.714607 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.714451    2573 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-c55st"
Jul 13 19:22:28.790085 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.789885    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-p47mr" podStartSLOduration=30.221249687 podStartE2EDuration="37.789833403s" totalImagesPullingTime="7.568583716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:21:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:28.378295875 +0000 UTC m=+45.892172483" watchObservedRunningTime="2026-07-13 19:22:28.789833403 +0000 UTC m=+46.303710010"
Jul 13 19:22:28.791034 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.790655    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ktrc2"]
Jul 13 19:22:28.806514 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.806473    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ktrc2"]
Jul 13 19:22:28.806743 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.806602    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:28.808997 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.808971    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 13 19:22:28.809144 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.809069    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 13 19:22:28.809197 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.809074    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dkwp7\""
Jul 13 19:22:28.924359 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.924322    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:28.924558 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.924413    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-crio-socket\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:28.924558 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.924439    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj4sn\" (UniqueName: \"kubernetes.io/projected/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-kube-api-access-pj4sn\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:28.924690 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.924561    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:28.924690 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:28.924615    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-data-volume\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:29.025428 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:29.025330    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-data-volume\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:29.025600 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:29.025468    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:29.025600 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:29.025544    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-crio-socket\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:29.025600 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:29.025570    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj4sn\" (UniqueName: \"kubernetes.io/projected/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-kube-api-access-pj4sn\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:29.025600 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:29.025590    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 19:22:29.025817 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:29.025653    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:29.025817 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:29.025679    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-kube-rbac-proxy-cm podName:492fd2c6-a58d-4a24-9a2f-1911b5747a0d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:29.525657807 +0000 UTC m=+47.039534391 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ktrc2" (UID: "492fd2c6-a58d-4a24-9a2f-1911b5747a0d") : configmap "kube-rbac-proxy" not found
Jul 13 19:22:29.025817 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:29.025694    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-crio-socket\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:29.025817 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:29.025674    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-data-volume\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:29.025817 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:29.025742    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 19:22:29.025817 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:29.025777    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-insights-runtime-extractor-tls podName:492fd2c6-a58d-4a24-9a2f-1911b5747a0d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:29.525769596 +0000 UTC m=+47.039646179 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ktrc2" (UID: "492fd2c6-a58d-4a24-9a2f-1911b5747a0d") : secret "insights-runtime-extractor-tls" not found
Jul 13 19:22:29.035270 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:29.035241    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj4sn\" (UniqueName: \"kubernetes.io/projected/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-kube-api-access-pj4sn\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:29.265144 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:29.265113    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/1.log"
Jul 13 19:22:29.265558 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:29.265541    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/0.log"
Jul 13 19:22:29.265622 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:29.265580    2573 generic.go:356] "Generic (PLEG): container finished" podID="319e53f6-7d9e-48f3-b9f6-001ada635739" containerID="ca1e4873ac518eb44d4994c313bcdb928dd648575d5a2847fa66fb17828ac1c9" exitCode=255
Jul 13 19:22:29.265716 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:29.265689    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-c55st" event={"ID":"319e53f6-7d9e-48f3-b9f6-001ada635739","Type":"ContainerDied","Data":"ca1e4873ac518eb44d4994c313bcdb928dd648575d5a2847fa66fb17828ac1c9"}
Jul 13 19:22:29.265767 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:29.265754    2573 scope.go:117] "RemoveContainer" containerID="cfbd9f3244800c0df2adc5e76a210e42022c068db14ef5c3ead14652936b80b5"
Jul 13 19:22:29.266010 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:29.265991    2573 scope.go:117] "RemoveContainer" containerID="ca1e4873ac518eb44d4994c313bcdb928dd648575d5a2847fa66fb17828ac1c9"
Jul 13 19:22:29.266376 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:29.266352    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-c55st_openshift-console-operator(319e53f6-7d9e-48f3-b9f6-001ada635739)\"" pod="openshift-console-operator/console-operator-7866497494-c55st" podUID="319e53f6-7d9e-48f3-b9f6-001ada635739"
Jul 13 19:22:29.531176 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:29.531133    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:29.531389 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:29.531241    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:29.531389 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:29.531324    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 19:22:29.531502 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:29.531405    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-insights-runtime-extractor-tls podName:492fd2c6-a58d-4a24-9a2f-1911b5747a0d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:30.531384196 +0000 UTC m=+48.045260780 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ktrc2" (UID: "492fd2c6-a58d-4a24-9a2f-1911b5747a0d") : secret "insights-runtime-extractor-tls" not found
Jul 13 19:22:29.531502 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:29.531333    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 19:22:29.531502 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:29.531457    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-kube-rbac-proxy-cm podName:492fd2c6-a58d-4a24-9a2f-1911b5747a0d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:30.531445382 +0000 UTC m=+48.045321965 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ktrc2" (UID: "492fd2c6-a58d-4a24-9a2f-1911b5747a0d") : configmap "kube-rbac-proxy" not found
Jul 13 19:22:30.270717 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:30.270691    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/1.log"
Jul 13 19:22:30.271089 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:30.271007    2573 scope.go:117] "RemoveContainer" containerID="ca1e4873ac518eb44d4994c313bcdb928dd648575d5a2847fa66fb17828ac1c9"
Jul 13 19:22:30.271175 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:30.271153    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-c55st_openshift-console-operator(319e53f6-7d9e-48f3-b9f6-001ada635739)\"" pod="openshift-console-operator/console-operator-7866497494-c55st" podUID="319e53f6-7d9e-48f3-b9f6-001ada635739"
Jul 13 19:22:30.540335 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:30.540233    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:30.540516 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:30.540365    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:30.540516 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:30.540386    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 19:22:30.540516 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:30.540479    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 19:22:30.540516 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:30.540487    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-insights-runtime-extractor-tls podName:492fd2c6-a58d-4a24-9a2f-1911b5747a0d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:32.540469711 +0000 UTC m=+50.054346295 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ktrc2" (UID: "492fd2c6-a58d-4a24-9a2f-1911b5747a0d") : secret "insights-runtime-extractor-tls" not found
Jul 13 19:22:30.540705 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:30.540535    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-kube-rbac-proxy-cm podName:492fd2c6-a58d-4a24-9a2f-1911b5747a0d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:32.540522156 +0000 UTC m=+50.054398740 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ktrc2" (UID: "492fd2c6-a58d-4a24-9a2f-1911b5747a0d") : configmap "kube-rbac-proxy" not found
Jul 13 19:22:31.073611 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:31.073581    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6kk2c_9d300bbe-5196-46b2-96a3-a35f690f0207/dns-node-resolver/0.log"
Jul 13 19:22:31.674506 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:31.674477    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-l6bw8_11a0b6c6-aae7-459a-9463-95459fc8ab61/node-ca/0.log"
Jul 13 19:22:32.475286 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:32.475255    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-59rng_95371626-7fce-48f7-a5ba-91cc14074e85/serve-healthcheck-canary/0.log"
Jul 13 19:22:32.557648 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:32.557610    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:32.557813 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:32.557721    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:32.557813 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:32.557763    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 19:22:32.557813 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:32.557789    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 19:22:32.557913 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:32.557843    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-insights-runtime-extractor-tls podName:492fd2c6-a58d-4a24-9a2f-1911b5747a0d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:36.55782165 +0000 UTC m=+54.071698235 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ktrc2" (UID: "492fd2c6-a58d-4a24-9a2f-1911b5747a0d") : secret "insights-runtime-extractor-tls" not found
Jul 13 19:22:32.557913 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:32.557880    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-kube-rbac-proxy-cm podName:492fd2c6-a58d-4a24-9a2f-1911b5747a0d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:36.557857344 +0000 UTC m=+54.071733932 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ktrc2" (UID: "492fd2c6-a58d-4a24-9a2f-1911b5747a0d") : configmap "kube-rbac-proxy" not found
Jul 13 19:22:34.272788 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:34.272732    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0fce8f4-676e-4fec-adf5-09b3fd26434d-metrics-tls\") pod \"dns-default-s9z7s\" (UID: \"a0fce8f4-676e-4fec-adf5-09b3fd26434d\") " pod="openshift-dns/dns-default-s9z7s"
Jul 13 19:22:34.273237 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:34.272855    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54bb1498-3f40-47ca-a33c-eaebcf72090b-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-dqtq2\" (UID: \"54bb1498-3f40-47ca-a33c-eaebcf72090b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-dqtq2"
Jul 13 19:22:34.273237 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:34.272896    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 19:22:34.273237 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:34.272944    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:34.273237 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:34.272956    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0fce8f4-676e-4fec-adf5-09b3fd26434d-metrics-tls podName:a0fce8f4-676e-4fec-adf5-09b3fd26434d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:50.272941123 +0000 UTC m=+67.786817707 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a0fce8f4-676e-4fec-adf5-09b3fd26434d-metrics-tls") pod "dns-default-s9z7s" (UID: "a0fce8f4-676e-4fec-adf5-09b3fd26434d") : secret "dns-default-metrics-tls" not found
Jul 13 19:22:34.273237 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:34.272999    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 13 19:22:34.273237 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:34.273025    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 19:22:34.273237 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:34.273040    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b77547c98-zzq67: secret "image-registry-tls" not found
Jul 13 19:22:34.273237 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:34.273059    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54bb1498-3f40-47ca-a33c-eaebcf72090b-networking-console-plugin-cert podName:54bb1498-3f40-47ca-a33c-eaebcf72090b nodeName:}" failed. No retries permitted until 2026-07-13 19:22:50.273043083 +0000 UTC m=+67.786919667 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/54bb1498-3f40-47ca-a33c-eaebcf72090b-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-dqtq2" (UID: "54bb1498-3f40-47ca-a33c-eaebcf72090b") : secret "networking-console-plugin-cert" not found
Jul 13 19:22:34.273237 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:34.273081    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls podName:b1572934-6111-4394-9114-396122b025e7 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:50.2730672 +0000 UTC m=+67.786943801 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls") pod "image-registry-5b77547c98-zzq67" (UID: "b1572934-6111-4394-9114-396122b025e7") : secret "image-registry-tls" not found
Jul 13 19:22:34.374004 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:34.373957    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51a39015-504a-4953-82e0-45eb6b25be9d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7556r\" (UID: \"51a39015-504a-4953-82e0-45eb6b25be9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7556r"
Jul 13 19:22:34.374187 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:34.374067    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a55203b2-d840-4319-9ced-26799e123698-service-ca-bundle\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:34.374187 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:34.374112    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0acc219-19c5-4387-a6d3-7c83cffb00df-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gr7gb\" (UID: \"c0acc219-19c5-4387-a6d3-7c83cffb00df\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb"
Jul 13 19:22:34.374187 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:34.374122    2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 19:22:34.374187 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:34.374150    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-metrics-certs\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:34.374187 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:34.374189    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51a39015-504a-4953-82e0-45eb6b25be9d-samples-operator-tls podName:51a39015-504a-4953-82e0-45eb6b25be9d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:50.37416767 +0000 UTC m=+67.888044257 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/51a39015-504a-4953-82e0-45eb6b25be9d-samples-operator-tls") pod "cluster-samples-operator-65db858894-7556r" (UID: "51a39015-504a-4953-82e0-45eb6b25be9d") : secret "samples-operator-tls" not found
Jul 13 19:22:34.374425 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:34.374211    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a55203b2-d840-4319-9ced-26799e123698-service-ca-bundle podName:a55203b2-d840-4319-9ced-26799e123698 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:50.374198057 +0000 UTC m=+67.888074641 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a55203b2-d840-4319-9ced-26799e123698-service-ca-bundle") pod "router-default-68dc6f48c8-qx2hp" (UID: "a55203b2-d840-4319-9ced-26799e123698") : configmap references non-existent config key: service-ca.crt
Jul 13 19:22:34.374425 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:34.374222    2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 13 19:22:34.374425 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:34.374256    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-metrics-certs podName:a55203b2-d840-4319-9ced-26799e123698 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:50.374246254 +0000 UTC m=+67.888122837 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-metrics-certs") pod "router-default-68dc6f48c8-qx2hp" (UID: "a55203b2-d840-4319-9ced-26799e123698") : secret "router-metrics-certs-default" not found
Jul 13 19:22:34.374425 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:34.374288    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 19:22:34.374425 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:34.374348    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0acc219-19c5-4387-a6d3-7c83cffb00df-cluster-monitoring-operator-tls podName:c0acc219-19c5-4387-a6d3-7c83cffb00df nodeName:}" failed. No retries permitted until 2026-07-13 19:22:50.37433309 +0000 UTC m=+67.888209681 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c0acc219-19c5-4387-a6d3-7c83cffb00df-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-gr7gb" (UID: "c0acc219-19c5-4387-a6d3-7c83cffb00df") : secret "cluster-monitoring-operator-tls" not found
Jul 13 19:22:36.594039 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:36.594004    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:36.594467 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:36.594208    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:36.594467 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:36.594325    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 19:22:36.594467 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:36.594378    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-insights-runtime-extractor-tls podName:492fd2c6-a58d-4a24-9a2f-1911b5747a0d nodeName:}" failed. No retries permitted until 2026-07-13 19:22:44.594363574 +0000 UTC m=+62.108240158 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ktrc2" (UID: "492fd2c6-a58d-4a24-9a2f-1911b5747a0d") : secret "insights-runtime-extractor-tls" not found
Jul 13 19:22:36.594576 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:36.594565    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:38.713533 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:38.713489    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-c55st"
Jul 13 19:22:38.714069 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:38.713598    2573 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-c55st"
Jul 13 19:22:38.714140 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:38.714079    2573 scope.go:117] "RemoveContainer" containerID="ca1e4873ac518eb44d4994c313bcdb928dd648575d5a2847fa66fb17828ac1c9"
Jul 13 19:22:39.297657 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:39.297605    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 19:22:39.298068 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:39.298050    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/1.log"
Jul 13 19:22:39.298186 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:39.298088    2573 generic.go:356] "Generic (PLEG): container finished" podID="319e53f6-7d9e-48f3-b9f6-001ada635739" containerID="bd54c84d163b0305a01b7ba5b48c1e24518cb17715644994d8f0f91f336a0862" exitCode=255
Jul 13 19:22:39.298186 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:39.298139    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-c55st" event={"ID":"319e53f6-7d9e-48f3-b9f6-001ada635739","Type":"ContainerDied","Data":"bd54c84d163b0305a01b7ba5b48c1e24518cb17715644994d8f0f91f336a0862"}
Jul 13 19:22:39.298291 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:39.298197    2573 scope.go:117] "RemoveContainer" containerID="ca1e4873ac518eb44d4994c313bcdb928dd648575d5a2847fa66fb17828ac1c9"
Jul 13 19:22:39.298517 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:39.298494    2573 scope.go:117] "RemoveContainer" containerID="bd54c84d163b0305a01b7ba5b48c1e24518cb17715644994d8f0f91f336a0862"
Jul 13 19:22:39.298840 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:39.298694    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-c55st_openshift-console-operator(319e53f6-7d9e-48f3-b9f6-001ada635739)\"" pod="openshift-console-operator/console-operator-7866497494-c55st" podUID="319e53f6-7d9e-48f3-b9f6-001ada635739"
Jul 13 19:22:40.303453 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:40.303425    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 19:22:43.190141 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:43.190109    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cj8b5"
Jul 13 19:22:44.667438 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:44.667404    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:44.669774 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:44.669751    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/492fd2c6-a58d-4a24-9a2f-1911b5747a0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktrc2\" (UID: \"492fd2c6-a58d-4a24-9a2f-1911b5747a0d\") " pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:44.720416 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:44.720385    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dkwp7\""
Jul 13 19:22:44.728709 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:44.728686    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ktrc2"
Jul 13 19:22:44.888089 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:44.888056    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ktrc2"]
Jul 13 19:22:44.891276 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:22:44.891244    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod492fd2c6_a58d_4a24_9a2f_1911b5747a0d.slice/crio-627f49032ae2e66d66d991b7bd68f3e88c2996fe788af733d0acf1e87338bb48 WatchSource:0}: Error finding container 627f49032ae2e66d66d991b7bd68f3e88c2996fe788af733d0acf1e87338bb48: Status 404 returned error can't find the container with id 627f49032ae2e66d66d991b7bd68f3e88c2996fe788af733d0acf1e87338bb48
Jul 13 19:22:45.319754 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:45.319713    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ktrc2" event={"ID":"492fd2c6-a58d-4a24-9a2f-1911b5747a0d","Type":"ContainerStarted","Data":"627f49032ae2e66d66d991b7bd68f3e88c2996fe788af733d0acf1e87338bb48"}
Jul 13 19:22:48.608493 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:48.608446    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret\") pod \"global-pull-secret-syncer-xxrt7\" (UID: \"690fc787-2061-47e9-8a05-7e596f4ddf0d\") " pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:22:48.611448 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:48.611422    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 13 19:22:48.621777 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:48.621756    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/690fc787-2061-47e9-8a05-7e596f4ddf0d-original-pull-secret\") pod \"global-pull-secret-syncer-xxrt7\" (UID: \"690fc787-2061-47e9-8a05-7e596f4ddf0d\") " pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:22:48.709478 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:48.709443    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs\") pod \"network-metrics-daemon-hjx4z\" (UID: \"fff12175-e77b-4ec7-b0ca-eb4b45145f76\") " pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:22:48.709673 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:48.709495    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlxgj\" (UniqueName: \"kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj\") pod \"network-check-target-2q22b\" (UID: \"cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8\") " pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:22:48.712079 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:48.712053    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlxgj\" (UniqueName: \"kubernetes.io/projected/cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8-kube-api-access-dlxgj\") pod \"network-check-target-2q22b\" (UID: \"cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8\") " pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:22:48.713399 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:48.713381    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-c55st"
Jul 13 19:22:48.713504 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:48.713426    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 13 19:22:48.713504 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:48.713430    2573 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-c55st"
Jul 13 19:22:48.713789 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:48.713774    2573 scope.go:117] "RemoveContainer" containerID="bd54c84d163b0305a01b7ba5b48c1e24518cb17715644994d8f0f91f336a0862"
Jul 13 19:22:48.713954 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:48.713934    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-c55st_openshift-console-operator(319e53f6-7d9e-48f3-b9f6-001ada635739)\"" pod="openshift-console-operator/console-operator-7866497494-c55st" podUID="319e53f6-7d9e-48f3-b9f6-001ada635739"
Jul 13 19:22:48.721762 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:48.721741    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fff12175-e77b-4ec7-b0ca-eb4b45145f76-metrics-certs\") pod \"network-metrics-daemon-hjx4z\" (UID: \"fff12175-e77b-4ec7-b0ca-eb4b45145f76\") " pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:22:48.855819 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:48.855788    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ztp4q\""
Jul 13 19:22:48.861750 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:48.861681    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fp668\""
Jul 13 19:22:48.863563 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:48.863546    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:22:48.865060 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:48.865042    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxrt7"
Jul 13 19:22:48.870671 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:48.870650    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjx4z"
Jul 13 19:22:49.005056 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:49.005000    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xxrt7"]
Jul 13 19:22:49.008121 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:22:49.008094    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod690fc787_2061_47e9_8a05_7e596f4ddf0d.slice/crio-811ad59fa17e54c545992d8ad9ff3ab1f911885c4565dfa4c1fe03b173f4a0e9 WatchSource:0}: Error finding container 811ad59fa17e54c545992d8ad9ff3ab1f911885c4565dfa4c1fe03b173f4a0e9: Status 404 returned error can't find the container with id 811ad59fa17e54c545992d8ad9ff3ab1f911885c4565dfa4c1fe03b173f4a0e9
Jul 13 19:22:49.021630 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:49.021558    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hjx4z"]
Jul 13 19:22:49.023783 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:22:49.023760    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfff12175_e77b_4ec7_b0ca_eb4b45145f76.slice/crio-48305cd9e215e8738181200517f28bf5261214bf6f8b9193376fdc08ae95cd10 WatchSource:0}: Error finding container 48305cd9e215e8738181200517f28bf5261214bf6f8b9193376fdc08ae95cd10: Status 404 returned error can't find the container with id 48305cd9e215e8738181200517f28bf5261214bf6f8b9193376fdc08ae95cd10
Jul 13 19:22:49.038496 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:49.038471    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2q22b"]
Jul 13 19:22:49.041500 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:22:49.041472    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf8ae4bd_c2f1_456e_9cf7_fcf5eef5e5e8.slice/crio-03b2d6ab24480ead38259c57322fe5bc6ad5cdf3322fab5346fbe04065ab365a WatchSource:0}: Error finding container 03b2d6ab24480ead38259c57322fe5bc6ad5cdf3322fab5346fbe04065ab365a: Status 404 returned error can't find the container with id 03b2d6ab24480ead38259c57322fe5bc6ad5cdf3322fab5346fbe04065ab365a
Jul 13 19:22:49.334482 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:49.334439    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xxrt7" event={"ID":"690fc787-2061-47e9-8a05-7e596f4ddf0d","Type":"ContainerStarted","Data":"811ad59fa17e54c545992d8ad9ff3ab1f911885c4565dfa4c1fe03b173f4a0e9"}
Jul 13 19:22:49.335936 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:49.335909    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ktrc2" event={"ID":"492fd2c6-a58d-4a24-9a2f-1911b5747a0d","Type":"ContainerStarted","Data":"f88293bca860e0e66a6e9bd2d7e3cd66bf91e985deb8714527d3c18e229d985a"}
Jul 13 19:22:49.336938 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:49.336915    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hjx4z" event={"ID":"fff12175-e77b-4ec7-b0ca-eb4b45145f76","Type":"ContainerStarted","Data":"48305cd9e215e8738181200517f28bf5261214bf6f8b9193376fdc08ae95cd10"}
Jul 13 19:22:49.338324 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:49.338302    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2q22b" event={"ID":"cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8","Type":"ContainerStarted","Data":"12976040fe052484262af61d80e2936241ea610c84110e2f3c72fb469fc5eab8"}
Jul 13 19:22:49.338426 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:49.338342    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2q22b" event={"ID":"cf8ae4bd-c2f1-456e-9cf7-fcf5eef5e5e8","Type":"ContainerStarted","Data":"03b2d6ab24480ead38259c57322fe5bc6ad5cdf3322fab5346fbe04065ab365a"}
Jul 13 19:22:49.338426 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:49.338376    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:22:49.338727 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:49.338713    2573 scope.go:117] "RemoveContainer" containerID="bd54c84d163b0305a01b7ba5b48c1e24518cb17715644994d8f0f91f336a0862"
Jul 13 19:22:49.338895 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:22:49.338875    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-c55st_openshift-console-operator(319e53f6-7d9e-48f3-b9f6-001ada635739)\"" pod="openshift-console-operator/console-operator-7866497494-c55st" podUID="319e53f6-7d9e-48f3-b9f6-001ada635739"
Jul 13 19:22:49.354454 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:49.354396    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2q22b" podStartSLOduration=66.354377314 podStartE2EDuration="1m6.354377314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:21:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:49.353776279 +0000 UTC m=+66.867652885" watchObservedRunningTime="2026-07-13 19:22:49.354377314 +0000 UTC m=+66.868253923"
Jul 13 19:22:50.326031 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.325991    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54bb1498-3f40-47ca-a33c-eaebcf72090b-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-dqtq2\" (UID: \"54bb1498-3f40-47ca-a33c-eaebcf72090b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-dqtq2"
Jul 13 19:22:50.326435 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.326091    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:50.326435 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.326162    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0fce8f4-676e-4fec-adf5-09b3fd26434d-metrics-tls\") pod \"dns-default-s9z7s\" (UID: \"a0fce8f4-676e-4fec-adf5-09b3fd26434d\") " pod="openshift-dns/dns-default-s9z7s"
Jul 13 19:22:50.329275 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.329196    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0fce8f4-676e-4fec-adf5-09b3fd26434d-metrics-tls\") pod \"dns-default-s9z7s\" (UID: \"a0fce8f4-676e-4fec-adf5-09b3fd26434d\") " pod="openshift-dns/dns-default-s9z7s"
Jul 13 19:22:50.329275 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.329264    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54bb1498-3f40-47ca-a33c-eaebcf72090b-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-dqtq2\" (UID: \"54bb1498-3f40-47ca-a33c-eaebcf72090b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-dqtq2"
Jul 13 19:22:50.329474 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.329298    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls\") pod \"image-registry-5b77547c98-zzq67\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") " pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:50.427436 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.427314    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0acc219-19c5-4387-a6d3-7c83cffb00df-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gr7gb\" (UID: \"c0acc219-19c5-4387-a6d3-7c83cffb00df\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb"
Jul 13 19:22:50.427436 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.427391    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-metrics-certs\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:50.427436 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.427428    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51a39015-504a-4953-82e0-45eb6b25be9d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7556r\" (UID: \"51a39015-504a-4953-82e0-45eb6b25be9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7556r"
Jul 13 19:22:50.427753 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.427505    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a55203b2-d840-4319-9ced-26799e123698-service-ca-bundle\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:50.428256 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.428230    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a55203b2-d840-4319-9ced-26799e123698-service-ca-bundle\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:50.430593 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.430198    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a55203b2-d840-4319-9ced-26799e123698-metrics-certs\") pod \"router-default-68dc6f48c8-qx2hp\" (UID: \"a55203b2-d840-4319-9ced-26799e123698\") " pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:50.430593 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.430547    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0acc219-19c5-4387-a6d3-7c83cffb00df-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gr7gb\" (UID: \"c0acc219-19c5-4387-a6d3-7c83cffb00df\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb"
Jul 13 19:22:50.431199 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.431174    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51a39015-504a-4953-82e0-45eb6b25be9d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7556r\" (UID: \"51a39015-504a-4953-82e0-45eb6b25be9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7556r"
Jul 13 19:22:50.492199 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.492166    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-25765\""
Jul 13 19:22:50.500609 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.500476    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:50.508784 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.508721    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mf9hk\""
Jul 13 19:22:50.516029 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.515994    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-s9z7s"
Jul 13 19:22:50.559494 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.559465    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-kqx5c\""
Jul 13 19:22:50.567491 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.567453    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-dqtq2"
Jul 13 19:22:50.571500 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.571279    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-pz9r6\""
Jul 13 19:22:50.579549 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.579525    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7556r"
Jul 13 19:22:50.602571 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.602532    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xr8pl\""
Jul 13 19:22:50.607873 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.607835    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-g8xzp\""
Jul 13 19:22:50.610178 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.610139    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb"
Jul 13 19:22:50.617752 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.617651    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:50.780700 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.780427    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b77547c98-zzq67"]
Jul 13 19:22:50.816031 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.815999    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-s9z7s"]
Jul 13 19:22:50.865318 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.865287    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7556r"]
Jul 13 19:22:50.898660 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.898611    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-dqtq2"]
Jul 13 19:22:50.903399 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:22:50.903027    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54bb1498_3f40_47ca_a33c_eaebcf72090b.slice/crio-550008bc837d6a13f092d6b181de72b090df84cfb267fe3321fbc36cbe1dce29 WatchSource:0}: Error finding container 550008bc837d6a13f092d6b181de72b090df84cfb267fe3321fbc36cbe1dce29: Status 404 returned error can't find the container with id 550008bc837d6a13f092d6b181de72b090df84cfb267fe3321fbc36cbe1dce29
Jul 13 19:22:50.923882 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.923855    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb"]
Jul 13 19:22:50.926662 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:22:50.926618    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0acc219_19c5_4387_a6d3_7c83cffb00df.slice/crio-8c7e6302672d1501cad687ecd34ca3812dc17c68f620b2cfecede891a974e73a WatchSource:0}: Error finding container 8c7e6302672d1501cad687ecd34ca3812dc17c68f620b2cfecede891a974e73a: Status 404 returned error can't find the container with id 8c7e6302672d1501cad687ecd34ca3812dc17c68f620b2cfecede891a974e73a
Jul 13 19:22:50.948580 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:50.948553    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-68dc6f48c8-qx2hp"]
Jul 13 19:22:50.951033 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:22:50.951007    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda55203b2_d840_4319_9ced_26799e123698.slice/crio-b68c0979b974d2d43a1ed712429f51082d013c48824579f41e920065eb5deae3 WatchSource:0}: Error finding container b68c0979b974d2d43a1ed712429f51082d013c48824579f41e920065eb5deae3: Status 404 returned error can't find the container with id b68c0979b974d2d43a1ed712429f51082d013c48824579f41e920065eb5deae3
Jul 13 19:22:51.350251 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:51.350193    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hjx4z" event={"ID":"fff12175-e77b-4ec7-b0ca-eb4b45145f76","Type":"ContainerStarted","Data":"479dea4eb2db710c127a3674699ae6ade5e3dd93b526d6eeded39e05176f1269"}
Jul 13 19:22:51.350732 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:51.350255    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hjx4z" event={"ID":"fff12175-e77b-4ec7-b0ca-eb4b45145f76","Type":"ContainerStarted","Data":"e67721b01f1df1fd4155ef1a0516ccb1a3e8cbdde7f5dec89261c40c8de3d20a"}
Jul 13 19:22:51.352918 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:51.352869    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-68dc6f48c8-qx2hp" event={"ID":"a55203b2-d840-4319-9ced-26799e123698","Type":"ContainerStarted","Data":"640bd49525d6facdd51ac9bc7dce420d8892a44e1e51e57481197957935910ad"}
Jul 13 19:22:51.353044 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:51.352940    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-68dc6f48c8-qx2hp" event={"ID":"a55203b2-d840-4319-9ced-26799e123698","Type":"ContainerStarted","Data":"b68c0979b974d2d43a1ed712429f51082d013c48824579f41e920065eb5deae3"}
Jul 13 19:22:51.354426 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:51.354397    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb" event={"ID":"c0acc219-19c5-4387-a6d3-7c83cffb00df","Type":"ContainerStarted","Data":"8c7e6302672d1501cad687ecd34ca3812dc17c68f620b2cfecede891a974e73a"}
Jul 13 19:22:51.355886 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:51.355861    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7556r" event={"ID":"51a39015-504a-4953-82e0-45eb6b25be9d","Type":"ContainerStarted","Data":"17770a3a765582e8225a51a1a73352dbbf8ad05b610591980c7e3be8c7a3a5ea"}
Jul 13 19:22:51.357301 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:51.357275    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-dqtq2" event={"ID":"54bb1498-3f40-47ca-a33c-eaebcf72090b","Type":"ContainerStarted","Data":"550008bc837d6a13f092d6b181de72b090df84cfb267fe3321fbc36cbe1dce29"}
Jul 13 19:22:51.358846 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:51.358818    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b77547c98-zzq67" event={"ID":"b1572934-6111-4394-9114-396122b025e7","Type":"ContainerStarted","Data":"4e31aa7c3b051fbe55506497b65964878fa49cf14c77ac8df974c9abcbe9bacc"}
Jul 13 19:22:51.358975 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:51.358876    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:22:51.358975 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:51.358898    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b77547c98-zzq67" event={"ID":"b1572934-6111-4394-9114-396122b025e7","Type":"ContainerStarted","Data":"f4c0db9bf924a65710541621e54873122f1a51c90096c7ab009601240a991649"}
Jul 13 19:22:51.360608 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:51.360586    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-s9z7s" event={"ID":"a0fce8f4-676e-4fec-adf5-09b3fd26434d","Type":"ContainerStarted","Data":"991ca847706ac8ba292c6fa1a6937828668089030072da85ccc08dc49ed98728"}
Jul 13 19:22:51.362683 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:51.362660    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ktrc2" event={"ID":"492fd2c6-a58d-4a24-9a2f-1911b5747a0d","Type":"ContainerStarted","Data":"9f7e4dfa52cb179bec24871301cc3dae7e33e7029336ca0cfa5705f5aa02e3f4"}
Jul 13 19:22:51.371520 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:51.371468    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hjx4z" podStartSLOduration=67.142141072 podStartE2EDuration="1m8.37145146s" totalImagesPullingTime="1.229310388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:21:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:51.370675846 +0000 UTC m=+68.884552450" watchObservedRunningTime="2026-07-13 19:22:51.37145146 +0000 UTC m=+68.885328067"
Jul 13 19:22:51.392427 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:51.392381    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-68dc6f48c8-qx2hp" podStartSLOduration=59.392365957 podStartE2EDuration="59.392365957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:21:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:51.391220994 +0000 UTC m=+68.905097624" watchObservedRunningTime="2026-07-13 19:22:51.392365957 +0000 UTC m=+68.906242540"
Jul 13 19:22:51.416077 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:51.416014    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b77547c98-zzq67" podStartSLOduration=68.415991848 podStartE2EDuration="1m8.415991848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:21:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:51.414293359 +0000 UTC m=+68.928169967" watchObservedRunningTime="2026-07-13 19:22:51.415991848 +0000 UTC m=+68.929868455"
Jul 13 19:22:51.618751 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:51.618659    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:51.621786 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:51.621760    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:52.369351 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:52.369258    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:52.371364 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:52.371140    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-68dc6f48c8-qx2hp"
Jul 13 19:22:58.394871 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:58.394835    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-s9z7s" event={"ID":"a0fce8f4-676e-4fec-adf5-09b3fd26434d","Type":"ContainerStarted","Data":"998134c60c996b595c45308882d7140f90a11135f45887b4a217fb66e30b24b1"}
Jul 13 19:22:58.395302 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:58.394916    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-s9z7s" event={"ID":"a0fce8f4-676e-4fec-adf5-09b3fd26434d","Type":"ContainerStarted","Data":"5960db4ec8bcea281c57f451a3e57e3ca60e357816a2da7466cfcbb0dc4936d1"}
Jul 13 19:22:58.395302 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:58.395124    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-s9z7s"
Jul 13 19:22:58.397308 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:58.397261    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xxrt7" event={"ID":"690fc787-2061-47e9-8a05-7e596f4ddf0d","Type":"ContainerStarted","Data":"e3f1446e9b7ed910d0f2ae520073fcf8504cb7682370ab7317974198a0469e92"}
Jul 13 19:22:58.399835 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:58.399809    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb" event={"ID":"c0acc219-19c5-4387-a6d3-7c83cffb00df","Type":"ContainerStarted","Data":"7ce3e39efc53d851bd202558947a71c7b49390b8e907ad29048215c40936fae0"}
Jul 13 19:22:58.402240 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:58.402137    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7556r" event={"ID":"51a39015-504a-4953-82e0-45eb6b25be9d","Type":"ContainerStarted","Data":"4cb9605acaa856b6e1cce06126771f7567748f5a39b0255328ea9affd5856e68"}
Jul 13 19:22:58.403836 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:58.403808    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-dqtq2" event={"ID":"54bb1498-3f40-47ca-a33c-eaebcf72090b","Type":"ContainerStarted","Data":"abf4d4405808c26816b43f70ff5ce8a6786ac1f71d09bab8cd3a20dc68d33de3"}
Jul 13 19:22:58.416119 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:58.416073    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-s9z7s" podStartSLOduration=33.292466659 podStartE2EDuration="40.416059292s" totalImagesPullingTime="7.123592633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:58.415674612 +0000 UTC m=+75.929551238" watchObservedRunningTime="2026-07-13 19:22:58.416059292 +0000 UTC m=+75.929935898"
Jul 13 19:22:58.438628 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:58.438559    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gr7gb" podStartSLOduration=59.411410564 podStartE2EDuration="1m6.438537697s" totalImagesPullingTime="7.027127133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:21:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:58.436883245 +0000 UTC m=+75.950759857" watchObservedRunningTime="2026-07-13 19:22:58.438537697 +0000 UTC m=+75.952414307"
Jul 13 19:22:58.461212 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:58.461127    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-dqtq2" podStartSLOduration=60.410315696 podStartE2EDuration="1m7.461110831s" totalImagesPullingTime="7.050795135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:21:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:58.460743304 +0000 UTC m=+75.974619938" watchObservedRunningTime="2026-07-13 19:22:58.461110831 +0000 UTC m=+75.974987437"
Jul 13 19:22:58.477724 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:58.477657    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xxrt7" podStartSLOduration=66.531624205 podStartE2EDuration="1m15.477612214s" totalImagesPullingTime="8.945988009s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:58.476684624 +0000 UTC m=+75.990561264" watchObservedRunningTime="2026-07-13 19:22:58.477612214 +0000 UTC m=+75.991488820"
Jul 13 19:22:59.413052 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:59.413002    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ktrc2" event={"ID":"492fd2c6-a58d-4a24-9a2f-1911b5747a0d","Type":"ContainerStarted","Data":"d28983204bb3291e0453a6cade39908efda1c28580969283ede3da592d14b856"}
Jul 13 19:22:59.414850 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:59.414821    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7556r" event={"ID":"51a39015-504a-4953-82e0-45eb6b25be9d","Type":"ContainerStarted","Data":"5c96be710bf877f139441b4a7bb28b00a6019b1b2bb231abd55fbcd48f5c56a4"}
Jul 13 19:22:59.431465 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:59.431411    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ktrc2" podStartSLOduration=18.197778279 podStartE2EDuration="31.431396479s" totalImagesPullingTime="13.2336182s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:22:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:59.430232558 +0000 UTC m=+76.944109192" watchObservedRunningTime="2026-07-13 19:22:59.431396479 +0000 UTC m=+76.945273085"
Jul 13 19:22:59.447407 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:22:59.447350    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7556r" podStartSLOduration=61.404706611 podStartE2EDuration="1m8.447336409s" totalImagesPullingTime="7.042629798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:21:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:59.446311781 +0000 UTC m=+76.960188388" watchObservedRunningTime="2026-07-13 19:22:59.447336409 +0000 UTC m=+76.961213098"
Jul 13 19:23:03.044174 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:03.044067    2573 scope.go:117] "RemoveContainer" containerID="bd54c84d163b0305a01b7ba5b48c1e24518cb17715644994d8f0f91f336a0862"
Jul 13 19:23:03.435512 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:03.435430    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 19:23:03.435677 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:03.435512    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-c55st" event={"ID":"319e53f6-7d9e-48f3-b9f6-001ada635739","Type":"ContainerStarted","Data":"3bf0a2e1636e50104c4954e8e21dd5bc6baaa2257675f73ab4c6e2ff615799b8"}
Jul 13 19:23:03.435831 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:03.435807    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-c55st"
Jul 13 19:23:03.441363 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:03.441340    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-c55st"
Jul 13 19:23:03.452322 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:03.452272    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-c55st" podStartSLOduration=64.852695914 podStartE2EDuration="1m12.452255627s" totalImagesPullingTime="7.599559713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:21:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:23:03.452230347 +0000 UTC m=+80.966106970" watchObservedRunningTime="2026-07-13 19:23:03.452255627 +0000 UTC m=+80.966132234"
Jul 13 19:23:03.573331 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:03.573294    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-jxg7k"]
Jul 13 19:23:03.578304 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:03.577828    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-jxg7k"
Jul 13 19:23:03.581088 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:03.581052    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 13 19:23:03.581088 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:03.581075    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 13 19:23:03.581307 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:03.581085    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-g4x9r\""
Jul 13 19:23:03.588550 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:03.588527    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-jxg7k"]
Jul 13 19:23:03.745486 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:03.745441    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28jcw\" (UniqueName: \"kubernetes.io/projected/1c3603c3-d25d-4ab4-9708-213b4c3aeb88-kube-api-access-28jcw\") pod \"downloads-68c9cbb497-jxg7k\" (UID: \"1c3603c3-d25d-4ab4-9708-213b4c3aeb88\") " pod="openshift-console/downloads-68c9cbb497-jxg7k"
Jul 13 19:23:03.846495 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:03.846454    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28jcw\" (UniqueName: \"kubernetes.io/projected/1c3603c3-d25d-4ab4-9708-213b4c3aeb88-kube-api-access-28jcw\") pod \"downloads-68c9cbb497-jxg7k\" (UID: \"1c3603c3-d25d-4ab4-9708-213b4c3aeb88\") " pod="openshift-console/downloads-68c9cbb497-jxg7k"
Jul 13 19:23:03.855430 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:03.855400    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28jcw\" (UniqueName: \"kubernetes.io/projected/1c3603c3-d25d-4ab4-9708-213b4c3aeb88-kube-api-access-28jcw\") pod \"downloads-68c9cbb497-jxg7k\" (UID: \"1c3603c3-d25d-4ab4-9708-213b4c3aeb88\") " pod="openshift-console/downloads-68c9cbb497-jxg7k"
Jul 13 19:23:03.889130 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:03.889088    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-jxg7k"
Jul 13 19:23:04.025547 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:04.025468    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-jxg7k"]
Jul 13 19:23:04.030943 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:23:04.030901    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c3603c3_d25d_4ab4_9708_213b4c3aeb88.slice/crio-9da28c89edcc96c18619130e25faf2f0f10a388f414c930540d71bfac41e6764 WatchSource:0}: Error finding container 9da28c89edcc96c18619130e25faf2f0f10a388f414c930540d71bfac41e6764: Status 404 returned error can't find the container with id 9da28c89edcc96c18619130e25faf2f0f10a388f414c930540d71bfac41e6764
Jul 13 19:23:04.440579 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:04.440545    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-jxg7k" event={"ID":"1c3603c3-d25d-4ab4-9708-213b4c3aeb88","Type":"ContainerStarted","Data":"9da28c89edcc96c18619130e25faf2f0f10a388f414c930540d71bfac41e6764"}
Jul 13 19:23:06.091598 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.091562    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xfkgj"]
Jul 13 19:23:06.095807 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.095781    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.098356 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.098333    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 13 19:23:06.098470 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.098380    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 13 19:23:06.098745 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.098725    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 13 19:23:06.098865 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.098785    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gdkc8\""
Jul 13 19:23:06.099188 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.099170    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 13 19:23:06.268016 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.267975    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29014e6c-7b56-4cba-8291-567036c8047b-sys\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.268209 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.268024    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/29014e6c-7b56-4cba-8291-567036c8047b-node-exporter-textfile\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.268209 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.268072    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/29014e6c-7b56-4cba-8291-567036c8047b-root\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.268209 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.268150    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/29014e6c-7b56-4cba-8291-567036c8047b-node-exporter-accelerators-collector-config\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.268209 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.268195    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29014e6c-7b56-4cba-8291-567036c8047b-metrics-client-ca\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.268391 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.268237    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxct5\" (UniqueName: \"kubernetes.io/projected/29014e6c-7b56-4cba-8291-567036c8047b-kube-api-access-qxct5\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.268391 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.268265    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/29014e6c-7b56-4cba-8291-567036c8047b-node-exporter-wtmp\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.268391 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.268325    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29014e6c-7b56-4cba-8291-567036c8047b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.268391 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.268375    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/29014e6c-7b56-4cba-8291-567036c8047b-node-exporter-tls\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.368942 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.368849    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29014e6c-7b56-4cba-8291-567036c8047b-sys\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.368942 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.368909    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/29014e6c-7b56-4cba-8291-567036c8047b-node-exporter-textfile\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.369159 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.368957    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/29014e6c-7b56-4cba-8291-567036c8047b-root\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.369159 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.368997    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/29014e6c-7b56-4cba-8291-567036c8047b-node-exporter-accelerators-collector-config\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.369159 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.369025    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/29014e6c-7b56-4cba-8291-567036c8047b-root\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.369159 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.368953    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29014e6c-7b56-4cba-8291-567036c8047b-sys\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.369159 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.369033    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29014e6c-7b56-4cba-8291-567036c8047b-metrics-client-ca\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.369159 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.369122    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxct5\" (UniqueName: \"kubernetes.io/projected/29014e6c-7b56-4cba-8291-567036c8047b-kube-api-access-qxct5\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.369159 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.369158    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/29014e6c-7b56-4cba-8291-567036c8047b-node-exporter-wtmp\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.369480 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.369192    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29014e6c-7b56-4cba-8291-567036c8047b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.369480 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.369221    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/29014e6c-7b56-4cba-8291-567036c8047b-node-exporter-tls\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.369480 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.369287    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/29014e6c-7b56-4cba-8291-567036c8047b-node-exporter-textfile\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.369480 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.369415    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/29014e6c-7b56-4cba-8291-567036c8047b-node-exporter-wtmp\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.369717 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.369609    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/29014e6c-7b56-4cba-8291-567036c8047b-node-exporter-accelerators-collector-config\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.369717 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.369671    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29014e6c-7b56-4cba-8291-567036c8047b-metrics-client-ca\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.372072 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.372048    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/29014e6c-7b56-4cba-8291-567036c8047b-node-exporter-tls\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.372457 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.372430    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29014e6c-7b56-4cba-8291-567036c8047b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.377309 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.377288    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxct5\" (UniqueName: \"kubernetes.io/projected/29014e6c-7b56-4cba-8291-567036c8047b-kube-api-access-qxct5\") pod \"node-exporter-xfkgj\" (UID: \"29014e6c-7b56-4cba-8291-567036c8047b\") " pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.407365 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.407336    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xfkgj"
Jul 13 19:23:06.419075 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:23:06.419036    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29014e6c_7b56_4cba_8291_567036c8047b.slice/crio-568977c666c19b2e03b34b72e237db51d44c945808498305889de288d7e28dc3 WatchSource:0}: Error finding container 568977c666c19b2e03b34b72e237db51d44c945808498305889de288d7e28dc3: Status 404 returned error can't find the container with id 568977c666c19b2e03b34b72e237db51d44c945808498305889de288d7e28dc3
Jul 13 19:23:06.451488 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:06.451451    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xfkgj" event={"ID":"29014e6c-7b56-4cba-8291-567036c8047b","Type":"ContainerStarted","Data":"568977c666c19b2e03b34b72e237db51d44c945808498305889de288d7e28dc3"}
Jul 13 19:23:08.417934 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:08.417902    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-s9z7s"
Jul 13 19:23:08.463466 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:08.463428    2573 generic.go:356] "Generic (PLEG): container finished" podID="29014e6c-7b56-4cba-8291-567036c8047b" containerID="870c0f5fa4fe1b5278fc41e99f808adfc55de63d5800bf8522dda15c78bc00a9" exitCode=0
Jul 13 19:23:08.463672 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:08.463479    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xfkgj" event={"ID":"29014e6c-7b56-4cba-8291-567036c8047b","Type":"ContainerDied","Data":"870c0f5fa4fe1b5278fc41e99f808adfc55de63d5800bf8522dda15c78bc00a9"}
Jul 13 19:23:09.470731 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:09.470698    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xfkgj" event={"ID":"29014e6c-7b56-4cba-8291-567036c8047b","Type":"ContainerStarted","Data":"f4358bb9638cee3779dc99a43a15ea24cb2423491a2fb33f570c34269aedb5ae"}
Jul 13 19:23:09.471201 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:09.470752    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xfkgj" event={"ID":"29014e6c-7b56-4cba-8291-567036c8047b","Type":"ContainerStarted","Data":"8f93f60244c024c4ea213449f2820dbd529695652ae30a38b8d753546319d412"}
Jul 13 19:23:10.505877 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:10.505824    2573 patch_prober.go:28] interesting pod/image-registry-5b77547c98-zzq67 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 13 19:23:10.506283 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:10.505892    2573 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5b77547c98-zzq67" podUID="b1572934-6111-4394-9114-396122b025e7" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:23:12.335383 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.335325    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xfkgj" podStartSLOduration=5.191219343 podStartE2EDuration="6.33530897s" totalImagesPullingTime="1.06637697s" totalInitContainerRuntime="77.712657ms" isStatefulPod=true podCreationTimestamp="2026-07-13 19:23:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:23:09.522829189 +0000 UTC m=+87.036705794" watchObservedRunningTime="2026-07-13 19:23:12.33530897 +0000 UTC m=+89.849185583"
Jul 13 19:23:12.335850 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.335836    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 19:23:12.338766 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.338749    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.341752 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.341718    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 13 19:23:12.341752 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.341726    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 13 19:23:12.342414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.342392    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 13 19:23:12.347674 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.345280    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 13 19:23:12.347674 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.345611    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 13 19:23:12.347674 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.345681    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 13 19:23:12.347674 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.345789    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 13 19:23:12.347674 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.346423    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 13 19:23:12.347674 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.346493    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 13 19:23:12.348258 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.348007    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 13 19:23:12.348258 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.348008    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4rc0ohl5kct91\""
Jul 13 19:23:12.348595 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.348246    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 13 19:23:12.349001 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.348942    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 13 19:23:12.349179 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.349157    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 13 19:23:12.350195 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.350172    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 13 19:23:12.350765 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.350746    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-n6nvj\""
Jul 13 19:23:12.354153 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.354136    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 13 19:23:12.358421 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.358399    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 19:23:12.373973 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.373951    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:23:12.521892 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.521854    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5373ef4e-030e-4489-8d3a-34bebd342e5a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.521892 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.521885    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.522127 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.521905    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.522127 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.521934    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.522127 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.522110    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.522272 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.522154    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.522272 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.522248    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-config\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.522344 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.522280    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.522344 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.522315    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.522344 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.522341    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.522479 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.522367    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.522479 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.522392    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-web-config\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.522479 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.522432    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.522603 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.522485    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.522603 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.522510    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.522603 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.522531    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.522603 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.522571    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.522603 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.522596    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5373ef4e-030e-4489-8d3a-34bebd342e5a-config-out\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.522869 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.522631    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf5hm\" (UniqueName: \"kubernetes.io/projected/5373ef4e-030e-4489-8d3a-34bebd342e5a-kube-api-access-bf5hm\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.522869 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.522721    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.624043 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.623956    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.624043 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.624024    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5373ef4e-030e-4489-8d3a-34bebd342e5a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.624254 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.624053    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.624254 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.624083    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.624254 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.624110    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.624254 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.624138    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.624254 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.624168    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.624254 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.624198    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-config\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.624254 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.624237    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.624574 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.624267    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.624574 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.624294    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.624574 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.624318    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.624574 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.624343    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-web-config\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.624574 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.624377    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.624574 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.624410    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.624574 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.624436    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.624574 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.624459    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.624574 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.624488    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.624574 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.624515    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5373ef4e-030e-4489-8d3a-34bebd342e5a-config-out\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.624574 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.624550    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf5hm\" (UniqueName: \"kubernetes.io/projected/5373ef4e-030e-4489-8d3a-34bebd342e5a-kube-api-access-bf5hm\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.625058 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.624981    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.625058 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.624982    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.625876 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.625807    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.626227 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.626201    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.627931 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.627741    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.627931 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.627762    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.627931 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.627767    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5373ef4e-030e-4489-8d3a-34bebd342e5a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.629021 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.628323    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.629021 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.628581    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.629021 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.628814    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-web-config\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.629294 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.629058    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.629294 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.629139    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.629819 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.629520    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.630381 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.630340    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.631074 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.631046    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.631291 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.631269    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5373ef4e-030e-4489-8d3a-34bebd342e5a-config-out\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.631498 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.631477    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.631853 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.631823    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.632170 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.632149    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-config\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.636076 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.636054    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf5hm\" (UniqueName: \"kubernetes.io/projected/5373ef4e-030e-4489-8d3a-34bebd342e5a-kube-api-access-bf5hm\") pod \"prometheus-k8s-0\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.650112 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.650087    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:12.814318 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:12.814270    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 19:23:12.816697 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:23:12.816662    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5373ef4e_030e_4489_8d3a_34bebd342e5a.slice/crio-69732450fcc88788dbe09da5f499a1b772f77839e1a3d410adcd802677997ce9 WatchSource:0}: Error finding container 69732450fcc88788dbe09da5f499a1b772f77839e1a3d410adcd802677997ce9: Status 404 returned error can't find the container with id 69732450fcc88788dbe09da5f499a1b772f77839e1a3d410adcd802677997ce9
Jul 13 19:23:13.491079 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:13.491028    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5373ef4e-030e-4489-8d3a-34bebd342e5a","Type":"ContainerStarted","Data":"69732450fcc88788dbe09da5f499a1b772f77839e1a3d410adcd802677997ce9"}
Jul 13 19:23:14.393656 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:14.393593    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b77547c98-zzq67"]
Jul 13 19:23:14.497257 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:14.497225    2573 generic.go:356] "Generic (PLEG): container finished" podID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerID="4baaebb2237deed41c9cfb7b25634b070d98f5e434325c162827f38dfe0fcd80" exitCode=0
Jul 13 19:23:14.497795 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:14.497321    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5373ef4e-030e-4489-8d3a-34bebd342e5a","Type":"ContainerDied","Data":"4baaebb2237deed41c9cfb7b25634b070d98f5e434325c162827f38dfe0fcd80"}
Jul 13 19:23:20.345005 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:20.344973    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2q22b"
Jul 13 19:23:22.534486 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:22.534439    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-jxg7k" event={"ID":"1c3603c3-d25d-4ab4-9708-213b4c3aeb88","Type":"ContainerStarted","Data":"2652c606f62d676036235ab54307e2d2be99007a507f2add5b4bf98c7d55e9e1"}
Jul 13 19:23:22.535023 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:22.534901    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-jxg7k"
Jul 13 19:23:22.551423 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:22.551387    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-jxg7k"
Jul 13 19:23:22.553213 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:22.553156    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-jxg7k" podStartSLOduration=1.842333266 podStartE2EDuration="19.553137019s" totalImagesPullingTime="17.710803753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:23:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:23:22.551694098 +0000 UTC m=+100.065570734" watchObservedRunningTime="2026-07-13 19:23:22.553137019 +0000 UTC m=+100.067013625"
Jul 13 19:23:25.553177 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:25.553134    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5373ef4e-030e-4489-8d3a-34bebd342e5a","Type":"ContainerStarted","Data":"5bd906bb9855745bc6cb595ef3b8a04f05e35b0d3f04469b9f5b2ef85f0c3193"}
Jul 13 19:23:25.553704 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:25.553594    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5373ef4e-030e-4489-8d3a-34bebd342e5a","Type":"ContainerStarted","Data":"a35df931cefe4b20beff04ef7de9328b10086a0dde2ad3bc8eb06b704f82c070"}
Jul 13 19:23:28.572295 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:28.572158    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5373ef4e-030e-4489-8d3a-34bebd342e5a","Type":"ContainerStarted","Data":"ec19df8189183fcedace126a306288461d25467887a002b2de4e443f5428dff9"}
Jul 13 19:23:28.572295 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:28.572283    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5373ef4e-030e-4489-8d3a-34bebd342e5a","Type":"ContainerStarted","Data":"4341e485494c37124f75b1f6c84cbab4582d918d5330d75b040bcbb40c089e6e"}
Jul 13 19:23:28.572817 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:28.572321    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5373ef4e-030e-4489-8d3a-34bebd342e5a","Type":"ContainerStarted","Data":"0c3c0ef6bc520df627d9bc5ff2bef73410750cf84cd873de58e7d4043af6b5a6"}
Jul 13 19:23:29.581044 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:29.580996    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5373ef4e-030e-4489-8d3a-34bebd342e5a","Type":"ContainerStarted","Data":"6b66d305804b81f1093cba4000058b21512338e8d522b1f5d72f3e7a3b59e2c2"}
Jul 13 19:23:29.616473 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:29.615948    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.409107592 podStartE2EDuration="17.615925379s" totalImagesPullingTime="14.188450027s" totalInitContainerRuntime="18.36776ms" isStatefulPod=false podCreationTimestamp="2026-07-13 19:23:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:23:29.614350675 +0000 UTC m=+107.128227309" watchObservedRunningTime="2026-07-13 19:23:29.615925379 +0000 UTC m=+107.129801986"
Jul 13 19:23:32.650974 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:32.650926    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:23:33.600615 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:33.600579    2573 generic.go:356] "Generic (PLEG): container finished" podID="c5945d50-bd96-4904-96ba-ea69aaef93d8" containerID="0504b3b32425e8056096a3b86123f992f42c40b13a9206ae40153b78acc6c585" exitCode=0
Jul 13 19:23:33.600804 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:33.600681    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7" event={"ID":"c5945d50-bd96-4904-96ba-ea69aaef93d8","Type":"ContainerDied","Data":"0504b3b32425e8056096a3b86123f992f42c40b13a9206ae40153b78acc6c585"}
Jul 13 19:23:33.601053 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:33.601017    2573 scope.go:117] "RemoveContainer" containerID="0504b3b32425e8056096a3b86123f992f42c40b13a9206ae40153b78acc6c585"
Jul 13 19:23:34.606552 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:34.606516    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-nnsb7" event={"ID":"c5945d50-bd96-4904-96ba-ea69aaef93d8","Type":"ContainerStarted","Data":"08e5b3bf87ab673bc681281f0b13a4a9c6257e6b066100fa63c1d6ae8fd32359"}
Jul 13 19:23:39.415940 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.415894    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5b77547c98-zzq67" podUID="b1572934-6111-4394-9114-396122b025e7" containerName="registry" containerID="cri-o://4e31aa7c3b051fbe55506497b65964878fa49cf14c77ac8df974c9abcbe9bacc" gracePeriod=30
Jul 13 19:23:39.627349 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.627319    2573 generic.go:356] "Generic (PLEG): container finished" podID="b1572934-6111-4394-9114-396122b025e7" containerID="4e31aa7c3b051fbe55506497b65964878fa49cf14c77ac8df974c9abcbe9bacc" exitCode=0
Jul 13 19:23:39.627500 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.627379    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b77547c98-zzq67" event={"ID":"b1572934-6111-4394-9114-396122b025e7","Type":"ContainerDied","Data":"4e31aa7c3b051fbe55506497b65964878fa49cf14c77ac8df974c9abcbe9bacc"}
Jul 13 19:23:39.697699 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.697671    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:23:39.769815 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.769778    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b1572934-6111-4394-9114-396122b025e7-registry-certificates\") pod \"b1572934-6111-4394-9114-396122b025e7\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") "
Jul 13 19:23:39.769815 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.769818    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b1572934-6111-4394-9114-396122b025e7-trusted-ca\") pod \"b1572934-6111-4394-9114-396122b025e7\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") "
Jul 13 19:23:39.770045 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.769846    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b1572934-6111-4394-9114-396122b025e7-ca-trust-extracted\") pod \"b1572934-6111-4394-9114-396122b025e7\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") "
Jul 13 19:23:39.770045 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.769866    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b1572934-6111-4394-9114-396122b025e7-installation-pull-secrets\") pod \"b1572934-6111-4394-9114-396122b025e7\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") "
Jul 13 19:23:39.770147 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.770053    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbgdv\" (UniqueName: \"kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-kube-api-access-zbgdv\") pod \"b1572934-6111-4394-9114-396122b025e7\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") "
Jul 13 19:23:39.770198 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.770159    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-bound-sa-token\") pod \"b1572934-6111-4394-9114-396122b025e7\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") "
Jul 13 19:23:39.770249 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.770218    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls\") pod \"b1572934-6111-4394-9114-396122b025e7\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") "
Jul 13 19:23:39.770299 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.770262    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1572934-6111-4394-9114-396122b025e7-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "b1572934-6111-4394-9114-396122b025e7" (UID: "b1572934-6111-4394-9114-396122b025e7"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:23:39.770299 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.770277    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b1572934-6111-4394-9114-396122b025e7-image-registry-private-configuration\") pod \"b1572934-6111-4394-9114-396122b025e7\" (UID: \"b1572934-6111-4394-9114-396122b025e7\") "
Jul 13 19:23:39.770299 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.770274    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1572934-6111-4394-9114-396122b025e7-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "b1572934-6111-4394-9114-396122b025e7" (UID: "b1572934-6111-4394-9114-396122b025e7"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:23:39.770551 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.770518    2573 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b1572934-6111-4394-9114-396122b025e7-registry-certificates\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:23:39.770704 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.770557    2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b1572934-6111-4394-9114-396122b025e7-trusted-ca\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:23:39.772602 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.772570    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "b1572934-6111-4394-9114-396122b025e7" (UID: "b1572934-6111-4394-9114-396122b025e7"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:23:39.772744 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.772674    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1572934-6111-4394-9114-396122b025e7-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "b1572934-6111-4394-9114-396122b025e7" (UID: "b1572934-6111-4394-9114-396122b025e7"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:23:39.772808 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.772730    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-kube-api-access-zbgdv" (OuterVolumeSpecName: "kube-api-access-zbgdv") pod "b1572934-6111-4394-9114-396122b025e7" (UID: "b1572934-6111-4394-9114-396122b025e7"). InnerVolumeSpecName "kube-api-access-zbgdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:23:39.772847 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.772829    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b1572934-6111-4394-9114-396122b025e7" (UID: "b1572934-6111-4394-9114-396122b025e7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:23:39.772902 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.772885    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1572934-6111-4394-9114-396122b025e7-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "b1572934-6111-4394-9114-396122b025e7" (UID: "b1572934-6111-4394-9114-396122b025e7"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:23:39.778310 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.778287    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1572934-6111-4394-9114-396122b025e7-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "b1572934-6111-4394-9114-396122b025e7" (UID: "b1572934-6111-4394-9114-396122b025e7"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:23:39.871381 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.871339    2573 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-bound-sa-token\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:23:39.871381 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.871372    2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-registry-tls\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:23:39.871381 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.871384    2573 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b1572934-6111-4394-9114-396122b025e7-image-registry-private-configuration\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:23:39.871381 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.871393    2573 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b1572934-6111-4394-9114-396122b025e7-ca-trust-extracted\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:23:39.871680 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.871403    2573 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b1572934-6111-4394-9114-396122b025e7-installation-pull-secrets\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:23:39.871680 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:39.871411    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbgdv\" (UniqueName: \"kubernetes.io/projected/b1572934-6111-4394-9114-396122b025e7-kube-api-access-zbgdv\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:23:40.633025 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:40.632997    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b77547c98-zzq67"
Jul 13 19:23:40.633025 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:40.633001    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b77547c98-zzq67" event={"ID":"b1572934-6111-4394-9114-396122b025e7","Type":"ContainerDied","Data":"f4c0db9bf924a65710541621e54873122f1a51c90096c7ab009601240a991649"}
Jul 13 19:23:40.633507 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:40.633067    2573 scope.go:117] "RemoveContainer" containerID="4e31aa7c3b051fbe55506497b65964878fa49cf14c77ac8df974c9abcbe9bacc"
Jul 13 19:23:40.653979 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:40.653954    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b77547c98-zzq67"]
Jul 13 19:23:40.657296 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:40.657274    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5b77547c98-zzq67"]
Jul 13 19:23:41.048029 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:41.047992    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1572934-6111-4394-9114-396122b025e7" path="/var/lib/kubelet/pods/b1572934-6111-4394-9114-396122b025e7/volumes"
Jul 13 19:23:43.205460 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:23:43.205427    2573 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6c9fcfe_e75f_4c3d_bbf0_baee21960fbc.slice/crio-conmon-40b6ea5faa011b30f77971afa7af7ea9a5d2356f7b9cd0656fb85840e37a50a3.scope\": RecentStats: unable to find data in memory cache]"
Jul 13 19:23:43.648339 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:43.648302    2573 generic.go:356] "Generic (PLEG): container finished" podID="f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc" containerID="40b6ea5faa011b30f77971afa7af7ea9a5d2356f7b9cd0656fb85840e37a50a3" exitCode=0
Jul 13 19:23:43.648502 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:43.648375    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-p47mr" event={"ID":"f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc","Type":"ContainerDied","Data":"40b6ea5faa011b30f77971afa7af7ea9a5d2356f7b9cd0656fb85840e37a50a3"}
Jul 13 19:23:43.648708 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:43.648695    2573 scope.go:117] "RemoveContainer" containerID="40b6ea5faa011b30f77971afa7af7ea9a5d2356f7b9cd0656fb85840e37a50a3"
Jul 13 19:23:44.653890 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:44.653855    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-p47mr" event={"ID":"f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc","Type":"ContainerStarted","Data":"59d9a4f86e13837c1e942532440b543a5514e8a9b01d05c388dbd20c7c465327"}
Jul 13 19:23:49.673650 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:49.673594    2573 generic.go:356] "Generic (PLEG): container finished" podID="fce914fd-6bde-48a0-9215-e66d18b67607" containerID="c9a6ba0cbd9134b5b30572f0a6840159f70128be2568e1e312377c055b553086" exitCode=0
Jul 13 19:23:49.674028 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:49.673661    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvc57" event={"ID":"fce914fd-6bde-48a0-9215-e66d18b67607","Type":"ContainerDied","Data":"c9a6ba0cbd9134b5b30572f0a6840159f70128be2568e1e312377c055b553086"}
Jul 13 19:23:49.674028 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:49.674005    2573 scope.go:117] "RemoveContainer" containerID="c9a6ba0cbd9134b5b30572f0a6840159f70128be2568e1e312377c055b553086"
Jul 13 19:23:50.679611 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:23:50.679569    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvc57" event={"ID":"fce914fd-6bde-48a0-9215-e66d18b67607","Type":"ContainerStarted","Data":"55b9d97a2d1671cc7fd1f4a81187b72444ae286279a51982abeaa9bea4b36286"}
Jul 13 19:24:12.650488 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:12.650449    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:12.666670 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:12.666618    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:12.779935 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:12.779891    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:30.730801 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:30.730768    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 19:24:30.731313 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:30.731100    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="prometheus" containerID="cri-o://a35df931cefe4b20beff04ef7de9328b10086a0dde2ad3bc8eb06b704f82c070" gracePeriod=600
Jul 13 19:24:30.731313 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:30.731122    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="kube-rbac-proxy-thanos" containerID="cri-o://6b66d305804b81f1093cba4000058b21512338e8d522b1f5d72f3e7a3b59e2c2" gracePeriod=600
Jul 13 19:24:30.731313 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:30.731146    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="kube-rbac-proxy-web" containerID="cri-o://4341e485494c37124f75b1f6c84cbab4582d918d5330d75b040bcbb40c089e6e" gracePeriod=600
Jul 13 19:24:30.731313 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:30.731170    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="kube-rbac-proxy" containerID="cri-o://ec19df8189183fcedace126a306288461d25467887a002b2de4e443f5428dff9" gracePeriod=600
Jul 13 19:24:30.731313 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:30.731146    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="thanos-sidecar" containerID="cri-o://0c3c0ef6bc520df627d9bc5ff2bef73410750cf84cd873de58e7d4043af6b5a6" gracePeriod=600
Jul 13 19:24:30.731313 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:30.731188    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="config-reloader" containerID="cri-o://5bd906bb9855745bc6cb595ef3b8a04f05e35b0d3f04469b9f5b2ef85f0c3193" gracePeriod=600
Jul 13 19:24:31.841605 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:31.841572    2573 generic.go:356] "Generic (PLEG): container finished" podID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerID="4341e485494c37124f75b1f6c84cbab4582d918d5330d75b040bcbb40c089e6e" exitCode=0
Jul 13 19:24:31.841605 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:31.841604    2573 generic.go:356] "Generic (PLEG): container finished" podID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerID="0c3c0ef6bc520df627d9bc5ff2bef73410750cf84cd873de58e7d4043af6b5a6" exitCode=0
Jul 13 19:24:31.841605 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:31.841614    2573 generic.go:356] "Generic (PLEG): container finished" podID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerID="5bd906bb9855745bc6cb595ef3b8a04f05e35b0d3f04469b9f5b2ef85f0c3193" exitCode=0
Jul 13 19:24:31.842085 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:31.841764    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5373ef4e-030e-4489-8d3a-34bebd342e5a","Type":"ContainerDied","Data":"4341e485494c37124f75b1f6c84cbab4582d918d5330d75b040bcbb40c089e6e"}
Jul 13 19:24:31.842085 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:31.841871    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5373ef4e-030e-4489-8d3a-34bebd342e5a","Type":"ContainerDied","Data":"0c3c0ef6bc520df627d9bc5ff2bef73410750cf84cd873de58e7d4043af6b5a6"}
Jul 13 19:24:31.842085 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:31.841920    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5373ef4e-030e-4489-8d3a-34bebd342e5a","Type":"ContainerDied","Data":"5bd906bb9855745bc6cb595ef3b8a04f05e35b0d3f04469b9f5b2ef85f0c3193"}
Jul 13 19:24:32.766070 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:24:32.766023    2573 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="a35df931cefe4b20beff04ef7de9328b10086a0dde2ad3bc8eb06b704f82c070" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Jul 13 19:24:32.767154 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:24:32.767115    2573 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="a35df931cefe4b20beff04ef7de9328b10086a0dde2ad3bc8eb06b704f82c070" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Jul 13 19:24:32.767995 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:24:32.767967    2573 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="a35df931cefe4b20beff04ef7de9328b10086a0dde2ad3bc8eb06b704f82c070" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Jul 13 19:24:32.768100 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:24:32.768006    2573 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-monitoring/prometheus-k8s-0" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="prometheus" probeResult="unknown"
Jul 13 19:24:32.849196 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:32.849158    2573 generic.go:356] "Generic (PLEG): container finished" podID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerID="6b66d305804b81f1093cba4000058b21512338e8d522b1f5d72f3e7a3b59e2c2" exitCode=0
Jul 13 19:24:32.849196 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:32.849188    2573 generic.go:356] "Generic (PLEG): container finished" podID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerID="ec19df8189183fcedace126a306288461d25467887a002b2de4e443f5428dff9" exitCode=0
Jul 13 19:24:32.849579 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:32.849231    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5373ef4e-030e-4489-8d3a-34bebd342e5a","Type":"ContainerDied","Data":"6b66d305804b81f1093cba4000058b21512338e8d522b1f5d72f3e7a3b59e2c2"}
Jul 13 19:24:32.849579 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:32.849307    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5373ef4e-030e-4489-8d3a-34bebd342e5a","Type":"ContainerDied","Data":"ec19df8189183fcedace126a306288461d25467887a002b2de4e443f5428dff9"}
Jul 13 19:24:37.765530 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:24:37.765479    2573 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="a35df931cefe4b20beff04ef7de9328b10086a0dde2ad3bc8eb06b704f82c070" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Jul 13 19:24:37.766552 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:24:37.766514    2573 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="a35df931cefe4b20beff04ef7de9328b10086a0dde2ad3bc8eb06b704f82c070" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Jul 13 19:24:37.767489 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:24:37.767449    2573 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="a35df931cefe4b20beff04ef7de9328b10086a0dde2ad3bc8eb06b704f82c070" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Jul 13 19:24:37.767591 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:24:37.767504    2573 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-monitoring/prometheus-k8s-0" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="prometheus" probeResult="unknown"
Jul 13 19:24:38.776471 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.776444    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:38.877195 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.877161    2573 generic.go:356] "Generic (PLEG): container finished" podID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerID="a35df931cefe4b20beff04ef7de9328b10086a0dde2ad3bc8eb06b704f82c070" exitCode=0
Jul 13 19:24:38.877381 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.877203    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5373ef4e-030e-4489-8d3a-34bebd342e5a","Type":"ContainerDied","Data":"a35df931cefe4b20beff04ef7de9328b10086a0dde2ad3bc8eb06b704f82c070"}
Jul 13 19:24:38.877381 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.877247    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:38.877381 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.877270    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5373ef4e-030e-4489-8d3a-34bebd342e5a","Type":"ContainerDied","Data":"69732450fcc88788dbe09da5f499a1b772f77839e1a3d410adcd802677997ce9"}
Jul 13 19:24:38.877381 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.877305    2573 scope.go:117] "RemoveContainer" containerID="6b66d305804b81f1093cba4000058b21512338e8d522b1f5d72f3e7a3b59e2c2"
Jul 13 19:24:38.884342 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.884287    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-configmap-kubelet-serving-ca-bundle\") pod \"5373ef4e-030e-4489-8d3a-34bebd342e5a\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") "
Jul 13 19:24:38.884342 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.884323    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-prometheus-k8s-tls\") pod \"5373ef4e-030e-4489-8d3a-34bebd342e5a\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") "
Jul 13 19:24:38.884342 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.884340    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-metrics-client-certs\") pod \"5373ef4e-030e-4489-8d3a-34bebd342e5a\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") "
Jul 13 19:24:38.884564 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.884359    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"5373ef4e-030e-4489-8d3a-34bebd342e5a\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") "
Jul 13 19:24:38.884564 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.884382    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-rulefiles-1\") pod \"5373ef4e-030e-4489-8d3a-34bebd342e5a\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") "
Jul 13 19:24:38.884564 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.884520    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-trusted-ca-bundle\") pod \"5373ef4e-030e-4489-8d3a-34bebd342e5a\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") "
Jul 13 19:24:38.884751 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.884567    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bf5hm\" (UniqueName: \"kubernetes.io/projected/5373ef4e-030e-4489-8d3a-34bebd342e5a-kube-api-access-bf5hm\") pod \"5373ef4e-030e-4489-8d3a-34bebd342e5a\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") "
Jul 13 19:24:38.884751 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.884599    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5373ef4e-030e-4489-8d3a-34bebd342e5a-config-out\") pod \"5373ef4e-030e-4489-8d3a-34bebd342e5a\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") "
Jul 13 19:24:38.884751 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.884652    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-thanos-prometheus-http-client-file\") pod \"5373ef4e-030e-4489-8d3a-34bebd342e5a\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") "
Jul 13 19:24:38.884751 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.884682    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-grpc-tls\") pod \"5373ef4e-030e-4489-8d3a-34bebd342e5a\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") "
Jul 13 19:24:38.884751 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.884714    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-configmap-metrics-client-ca\") pod \"5373ef4e-030e-4489-8d3a-34bebd342e5a\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") "
Jul 13 19:24:38.884999 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.884753    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-db\") pod \"5373ef4e-030e-4489-8d3a-34bebd342e5a\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") "
Jul 13 19:24:38.884999 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.884809    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-rulefiles-0\") pod \"5373ef4e-030e-4489-8d3a-34bebd342e5a\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") "
Jul 13 19:24:38.884999 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.884839    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-config\") pod \"5373ef4e-030e-4489-8d3a-34bebd342e5a\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") "
Jul 13 19:24:38.884999 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.884867    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-rulefiles-2\") pod \"5373ef4e-030e-4489-8d3a-34bebd342e5a\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") "
Jul 13 19:24:38.884999 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.884900    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-configmap-serving-certs-ca-bundle\") pod \"5373ef4e-030e-4489-8d3a-34bebd342e5a\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") "
Jul 13 19:24:38.884999 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.884927    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-web-config\") pod \"5373ef4e-030e-4489-8d3a-34bebd342e5a\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") "
Jul 13 19:24:38.884999 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.884955    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5373ef4e-030e-4489-8d3a-34bebd342e5a-tls-assets\") pod \"5373ef4e-030e-4489-8d3a-34bebd342e5a\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") "
Jul 13 19:24:38.884999 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.885000    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"5373ef4e-030e-4489-8d3a-34bebd342e5a\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") "
Jul 13 19:24:38.885442 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.885030    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-kube-rbac-proxy\") pod \"5373ef4e-030e-4489-8d3a-34bebd342e5a\" (UID: \"5373ef4e-030e-4489-8d3a-34bebd342e5a\") "
Jul 13 19:24:38.886944 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.886923    2573 scope.go:117] "RemoveContainer" containerID="ec19df8189183fcedace126a306288461d25467887a002b2de4e443f5428dff9"
Jul 13 19:24:38.887050 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.884750    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "5373ef4e-030e-4489-8d3a-34bebd342e5a" (UID: "5373ef4e-030e-4489-8d3a-34bebd342e5a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:24:38.887050 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.884788    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "5373ef4e-030e-4489-8d3a-34bebd342e5a" (UID: "5373ef4e-030e-4489-8d3a-34bebd342e5a"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:24:38.887050 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.885171    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "5373ef4e-030e-4489-8d3a-34bebd342e5a" (UID: "5373ef4e-030e-4489-8d3a-34bebd342e5a"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:24:38.887050 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.885817    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "5373ef4e-030e-4489-8d3a-34bebd342e5a" (UID: "5373ef4e-030e-4489-8d3a-34bebd342e5a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:24:38.887310 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.887058    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "5373ef4e-030e-4489-8d3a-34bebd342e5a" (UID: "5373ef4e-030e-4489-8d3a-34bebd342e5a"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:24:38.887310 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.887303    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "5373ef4e-030e-4489-8d3a-34bebd342e5a" (UID: "5373ef4e-030e-4489-8d3a-34bebd342e5a"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:24:38.888486 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.888350    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "5373ef4e-030e-4489-8d3a-34bebd342e5a" (UID: "5373ef4e-030e-4489-8d3a-34bebd342e5a"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:24:38.888486 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.888387    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "5373ef4e-030e-4489-8d3a-34bebd342e5a" (UID: "5373ef4e-030e-4489-8d3a-34bebd342e5a"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:24:38.888486 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.888385    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "5373ef4e-030e-4489-8d3a-34bebd342e5a" (UID: "5373ef4e-030e-4489-8d3a-34bebd342e5a"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:24:38.888486 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.888403    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5373ef4e-030e-4489-8d3a-34bebd342e5a-config-out" (OuterVolumeSpecName: "config-out") pod "5373ef4e-030e-4489-8d3a-34bebd342e5a" (UID: "5373ef4e-030e-4489-8d3a-34bebd342e5a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:24:38.888486 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.888448    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "5373ef4e-030e-4489-8d3a-34bebd342e5a" (UID: "5373ef4e-030e-4489-8d3a-34bebd342e5a"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:24:38.888486 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.888462    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "5373ef4e-030e-4489-8d3a-34bebd342e5a" (UID: "5373ef4e-030e-4489-8d3a-34bebd342e5a"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:24:38.888850 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.888676    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "5373ef4e-030e-4489-8d3a-34bebd342e5a" (UID: "5373ef4e-030e-4489-8d3a-34bebd342e5a"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:24:38.888850 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.888696    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "5373ef4e-030e-4489-8d3a-34bebd342e5a" (UID: "5373ef4e-030e-4489-8d3a-34bebd342e5a"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:24:38.889478 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.889452    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-config" (OuterVolumeSpecName: "config") pod "5373ef4e-030e-4489-8d3a-34bebd342e5a" (UID: "5373ef4e-030e-4489-8d3a-34bebd342e5a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:24:38.889588 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.889572    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5373ef4e-030e-4489-8d3a-34bebd342e5a-kube-api-access-bf5hm" (OuterVolumeSpecName: "kube-api-access-bf5hm") pod "5373ef4e-030e-4489-8d3a-34bebd342e5a" (UID: "5373ef4e-030e-4489-8d3a-34bebd342e5a"). InnerVolumeSpecName "kube-api-access-bf5hm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:24:38.889793 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.889769    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "5373ef4e-030e-4489-8d3a-34bebd342e5a" (UID: "5373ef4e-030e-4489-8d3a-34bebd342e5a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:24:38.890014 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.889989    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5373ef4e-030e-4489-8d3a-34bebd342e5a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5373ef4e-030e-4489-8d3a-34bebd342e5a" (UID: "5373ef4e-030e-4489-8d3a-34bebd342e5a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:24:38.890503 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.890479    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "5373ef4e-030e-4489-8d3a-34bebd342e5a" (UID: "5373ef4e-030e-4489-8d3a-34bebd342e5a"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:24:38.903386 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.903351    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-web-config" (OuterVolumeSpecName: "web-config") pod "5373ef4e-030e-4489-8d3a-34bebd342e5a" (UID: "5373ef4e-030e-4489-8d3a-34bebd342e5a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:24:38.910980 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.910961    2573 scope.go:117] "RemoveContainer" containerID="4341e485494c37124f75b1f6c84cbab4582d918d5330d75b040bcbb40c089e6e"
Jul 13 19:24:38.918153 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.918133    2573 scope.go:117] "RemoveContainer" containerID="0c3c0ef6bc520df627d9bc5ff2bef73410750cf84cd873de58e7d4043af6b5a6"
Jul 13 19:24:38.924849 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.924830    2573 scope.go:117] "RemoveContainer" containerID="5bd906bb9855745bc6cb595ef3b8a04f05e35b0d3f04469b9f5b2ef85f0c3193"
Jul 13 19:24:38.931630 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.931610    2573 scope.go:117] "RemoveContainer" containerID="a35df931cefe4b20beff04ef7de9328b10086a0dde2ad3bc8eb06b704f82c070"
Jul 13 19:24:38.938672 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.938646    2573 scope.go:117] "RemoveContainer" containerID="4baaebb2237deed41c9cfb7b25634b070d98f5e434325c162827f38dfe0fcd80"
Jul 13 19:24:38.945248 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.945230    2573 scope.go:117] "RemoveContainer" containerID="6b66d305804b81f1093cba4000058b21512338e8d522b1f5d72f3e7a3b59e2c2"
Jul 13 19:24:38.945518 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:24:38.945498    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b66d305804b81f1093cba4000058b21512338e8d522b1f5d72f3e7a3b59e2c2\": container with ID starting with 6b66d305804b81f1093cba4000058b21512338e8d522b1f5d72f3e7a3b59e2c2 not found: ID does not exist" containerID="6b66d305804b81f1093cba4000058b21512338e8d522b1f5d72f3e7a3b59e2c2"
Jul 13 19:24:38.945568 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.945530    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b66d305804b81f1093cba4000058b21512338e8d522b1f5d72f3e7a3b59e2c2"} err="failed to get container status \"6b66d305804b81f1093cba4000058b21512338e8d522b1f5d72f3e7a3b59e2c2\": rpc error: code = NotFound desc = could not find container \"6b66d305804b81f1093cba4000058b21512338e8d522b1f5d72f3e7a3b59e2c2\": container with ID starting with 6b66d305804b81f1093cba4000058b21512338e8d522b1f5d72f3e7a3b59e2c2 not found: ID does not exist"
Jul 13 19:24:38.945568 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.945566    2573 scope.go:117] "RemoveContainer" containerID="ec19df8189183fcedace126a306288461d25467887a002b2de4e443f5428dff9"
Jul 13 19:24:38.945837 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:24:38.945816    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec19df8189183fcedace126a306288461d25467887a002b2de4e443f5428dff9\": container with ID starting with ec19df8189183fcedace126a306288461d25467887a002b2de4e443f5428dff9 not found: ID does not exist" containerID="ec19df8189183fcedace126a306288461d25467887a002b2de4e443f5428dff9"
Jul 13 19:24:38.945897 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.945843    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec19df8189183fcedace126a306288461d25467887a002b2de4e443f5428dff9"} err="failed to get container status \"ec19df8189183fcedace126a306288461d25467887a002b2de4e443f5428dff9\": rpc error: code = NotFound desc = could not find container \"ec19df8189183fcedace126a306288461d25467887a002b2de4e443f5428dff9\": container with ID starting with ec19df8189183fcedace126a306288461d25467887a002b2de4e443f5428dff9 not found: ID does not exist"
Jul 13 19:24:38.945897 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.945862    2573 scope.go:117] "RemoveContainer" containerID="4341e485494c37124f75b1f6c84cbab4582d918d5330d75b040bcbb40c089e6e"
Jul 13 19:24:38.946115 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:24:38.946096    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4341e485494c37124f75b1f6c84cbab4582d918d5330d75b040bcbb40c089e6e\": container with ID starting with 4341e485494c37124f75b1f6c84cbab4582d918d5330d75b040bcbb40c089e6e not found: ID does not exist" containerID="4341e485494c37124f75b1f6c84cbab4582d918d5330d75b040bcbb40c089e6e"
Jul 13 19:24:38.946150 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.946125    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4341e485494c37124f75b1f6c84cbab4582d918d5330d75b040bcbb40c089e6e"} err="failed to get container status \"4341e485494c37124f75b1f6c84cbab4582d918d5330d75b040bcbb40c089e6e\": rpc error: code = NotFound desc = could not find container \"4341e485494c37124f75b1f6c84cbab4582d918d5330d75b040bcbb40c089e6e\": container with ID starting with 4341e485494c37124f75b1f6c84cbab4582d918d5330d75b040bcbb40c089e6e not found: ID does not exist"
Jul 13 19:24:38.946150 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.946140    2573 scope.go:117] "RemoveContainer" containerID="0c3c0ef6bc520df627d9bc5ff2bef73410750cf84cd873de58e7d4043af6b5a6"
Jul 13 19:24:38.946370 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:24:38.946352    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c3c0ef6bc520df627d9bc5ff2bef73410750cf84cd873de58e7d4043af6b5a6\": container with ID starting with 0c3c0ef6bc520df627d9bc5ff2bef73410750cf84cd873de58e7d4043af6b5a6 not found: ID does not exist" containerID="0c3c0ef6bc520df627d9bc5ff2bef73410750cf84cd873de58e7d4043af6b5a6"
Jul 13 19:24:38.946425 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.946382    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c3c0ef6bc520df627d9bc5ff2bef73410750cf84cd873de58e7d4043af6b5a6"} err="failed to get container status \"0c3c0ef6bc520df627d9bc5ff2bef73410750cf84cd873de58e7d4043af6b5a6\": rpc error: code = NotFound desc = could not find container \"0c3c0ef6bc520df627d9bc5ff2bef73410750cf84cd873de58e7d4043af6b5a6\": container with ID starting with 0c3c0ef6bc520df627d9bc5ff2bef73410750cf84cd873de58e7d4043af6b5a6 not found: ID does not exist"
Jul 13 19:24:38.946425 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.946397    2573 scope.go:117] "RemoveContainer" containerID="5bd906bb9855745bc6cb595ef3b8a04f05e35b0d3f04469b9f5b2ef85f0c3193"
Jul 13 19:24:38.946607 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:24:38.946591    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bd906bb9855745bc6cb595ef3b8a04f05e35b0d3f04469b9f5b2ef85f0c3193\": container with ID starting with 5bd906bb9855745bc6cb595ef3b8a04f05e35b0d3f04469b9f5b2ef85f0c3193 not found: ID does not exist" containerID="5bd906bb9855745bc6cb595ef3b8a04f05e35b0d3f04469b9f5b2ef85f0c3193"
Jul 13 19:24:38.946671 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.946610    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bd906bb9855745bc6cb595ef3b8a04f05e35b0d3f04469b9f5b2ef85f0c3193"} err="failed to get container status \"5bd906bb9855745bc6cb595ef3b8a04f05e35b0d3f04469b9f5b2ef85f0c3193\": rpc error: code = NotFound desc = could not find container \"5bd906bb9855745bc6cb595ef3b8a04f05e35b0d3f04469b9f5b2ef85f0c3193\": container with ID starting with 5bd906bb9855745bc6cb595ef3b8a04f05e35b0d3f04469b9f5b2ef85f0c3193 not found: ID does not exist"
Jul 13 19:24:38.946671 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.946623    2573 scope.go:117] "RemoveContainer" containerID="a35df931cefe4b20beff04ef7de9328b10086a0dde2ad3bc8eb06b704f82c070"
Jul 13 19:24:38.946852 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:24:38.946834    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a35df931cefe4b20beff04ef7de9328b10086a0dde2ad3bc8eb06b704f82c070\": container with ID starting with a35df931cefe4b20beff04ef7de9328b10086a0dde2ad3bc8eb06b704f82c070 not found: ID does not exist" containerID="a35df931cefe4b20beff04ef7de9328b10086a0dde2ad3bc8eb06b704f82c070"
Jul 13 19:24:38.946893 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.946854    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a35df931cefe4b20beff04ef7de9328b10086a0dde2ad3bc8eb06b704f82c070"} err="failed to get container status \"a35df931cefe4b20beff04ef7de9328b10086a0dde2ad3bc8eb06b704f82c070\": rpc error: code = NotFound desc = could not find container \"a35df931cefe4b20beff04ef7de9328b10086a0dde2ad3bc8eb06b704f82c070\": container with ID starting with a35df931cefe4b20beff04ef7de9328b10086a0dde2ad3bc8eb06b704f82c070 not found: ID does not exist"
Jul 13 19:24:38.946893 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.946866    2573 scope.go:117] "RemoveContainer" containerID="4baaebb2237deed41c9cfb7b25634b070d98f5e434325c162827f38dfe0fcd80"
Jul 13 19:24:38.947078 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:24:38.947064    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4baaebb2237deed41c9cfb7b25634b070d98f5e434325c162827f38dfe0fcd80\": container with ID starting with 4baaebb2237deed41c9cfb7b25634b070d98f5e434325c162827f38dfe0fcd80 not found: ID does not exist" containerID="4baaebb2237deed41c9cfb7b25634b070d98f5e434325c162827f38dfe0fcd80"
Jul 13 19:24:38.947118 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.947081    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4baaebb2237deed41c9cfb7b25634b070d98f5e434325c162827f38dfe0fcd80"} err="failed to get container status \"4baaebb2237deed41c9cfb7b25634b070d98f5e434325c162827f38dfe0fcd80\": rpc error: code = NotFound desc = could not find container \"4baaebb2237deed41c9cfb7b25634b070d98f5e434325c162827f38dfe0fcd80\": container with ID starting with 4baaebb2237deed41c9cfb7b25634b070d98f5e434325c162827f38dfe0fcd80 not found: ID does not exist"
Jul 13 19:24:38.986175 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.986140    2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:24:38.986175 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.986167    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bf5hm\" (UniqueName: \"kubernetes.io/projected/5373ef4e-030e-4489-8d3a-34bebd342e5a-kube-api-access-bf5hm\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:24:38.986175 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.986180    2573 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5373ef4e-030e-4489-8d3a-34bebd342e5a-config-out\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:24:38.986364 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.986190    2573 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:24:38.986364 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.986198    2573 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-grpc-tls\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:24:38.986364 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.986206    2573 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-configmap-metrics-client-ca\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:24:38.986364 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.986217    2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-db\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:24:38.986364 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.986225    2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:24:38.986364 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.986234    2573 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-config\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:24:38.986364 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.986243    2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:24:38.986364 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.986252    2573 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:24:38.986364 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.986261    2573 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-web-config\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:24:38.986364 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.986269    2573 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5373ef4e-030e-4489-8d3a-34bebd342e5a-tls-assets\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:24:38.986364 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.986277    2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:24:38.986364 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.986285    2573 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-kube-rbac-proxy\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:24:38.986364 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.986294    2573 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:24:38.986364 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.986303    2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:24:38.986364 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.986312    2573 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-metrics-client-certs\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:24:38.986364 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.986321    2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5373ef4e-030e-4489-8d3a-34bebd342e5a-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:24:38.986364 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:38.986329    2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5373ef4e-030e-4489-8d3a-34bebd342e5a-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:24:39.201274 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.201231    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 19:24:39.205838 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.205810    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 19:24:39.231353 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231321    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 19:24:39.231658 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231622    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="thanos-sidecar"
Jul 13 19:24:39.231658 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231649    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="thanos-sidecar"
Jul 13 19:24:39.231759 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231664    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1572934-6111-4394-9114-396122b025e7" containerName="registry"
Jul 13 19:24:39.231759 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231671    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1572934-6111-4394-9114-396122b025e7" containerName="registry"
Jul 13 19:24:39.231759 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231680    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="kube-rbac-proxy-web"
Jul 13 19:24:39.231759 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231686    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="kube-rbac-proxy-web"
Jul 13 19:24:39.231759 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231698    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="prometheus"
Jul 13 19:24:39.231759 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231703    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="prometheus"
Jul 13 19:24:39.231759 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231710    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="config-reloader"
Jul 13 19:24:39.231759 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231716    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="config-reloader"
Jul 13 19:24:39.231759 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231724    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="init-config-reloader"
Jul 13 19:24:39.231759 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231729    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="init-config-reloader"
Jul 13 19:24:39.231759 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231737    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="kube-rbac-proxy-thanos"
Jul 13 19:24:39.231759 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231742    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="kube-rbac-proxy-thanos"
Jul 13 19:24:39.231759 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231748    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="kube-rbac-proxy"
Jul 13 19:24:39.231759 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231754    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="kube-rbac-proxy"
Jul 13 19:24:39.232176 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231812    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="thanos-sidecar"
Jul 13 19:24:39.232176 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231820    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="kube-rbac-proxy-thanos"
Jul 13 19:24:39.232176 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231826    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1572934-6111-4394-9114-396122b025e7" containerName="registry"
Jul 13 19:24:39.232176 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231832    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="config-reloader"
Jul 13 19:24:39.232176 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231838    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="prometheus"
Jul 13 19:24:39.232176 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231845    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="kube-rbac-proxy-web"
Jul 13 19:24:39.232176 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.231851    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" containerName="kube-rbac-proxy"
Jul 13 19:24:39.236188 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.236170    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.238629 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.238603    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 13 19:24:39.238769 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.238603    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 13 19:24:39.239094 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.239070    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 13 19:24:39.239094 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.239073    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 13 19:24:39.239094 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.239077    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 13 19:24:39.239294 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.239119    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 13 19:24:39.239294 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.239135    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 13 19:24:39.239294 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.239140    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 13 19:24:39.239516 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.239499    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 13 19:24:39.239564 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.239534    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 13 19:24:39.239604 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.239589    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 13 19:24:39.239673 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.239606    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-n6nvj\""
Jul 13 19:24:39.239971 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.239938    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 13 19:24:39.240051 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.239997    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4rc0ohl5kct91\""
Jul 13 19:24:39.240667 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.240650    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 13 19:24:39.243543 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.243411    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 13 19:24:39.246592 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.246572    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 13 19:24:39.252229 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.252203    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 19:24:39.389714 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.389665    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17fa1a59-da82-416c-904c-61c8c9a8b4f6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.389714 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.389720    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-config\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.389959 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.389792    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/17fa1a59-da82-416c-904c-61c8c9a8b4f6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.389959 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.389819    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/17fa1a59-da82-416c-904c-61c8c9a8b4f6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.389959 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.389840    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.389959 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.389861    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.389959 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.389892    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/17fa1a59-da82-416c-904c-61c8c9a8b4f6-config-out\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.389959 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.389917    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-web-config\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.389959 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.389934    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.389959 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.389958    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17fa1a59-da82-416c-904c-61c8c9a8b4f6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.390223 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.389988    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.390223 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.390006    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.390223 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.390020    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/17fa1a59-da82-416c-904c-61c8c9a8b4f6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.390223 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.390041    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17fa1a59-da82-416c-904c-61c8c9a8b4f6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.390223 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.390059    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17fa1a59-da82-416c-904c-61c8c9a8b4f6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.390223 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.390075    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/17fa1a59-da82-416c-904c-61c8c9a8b4f6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.390223 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.390094    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.390223 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.390141    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/17fa1a59-da82-416c-904c-61c8c9a8b4f6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.390223 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.390159    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.390223 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.390176    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kts78\" (UniqueName: \"kubernetes.io/projected/17fa1a59-da82-416c-904c-61c8c9a8b4f6-kube-api-access-kts78\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.491087 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.490988    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17fa1a59-da82-416c-904c-61c8c9a8b4f6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.491087 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.491042    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-config\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.491087 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.491063    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/17fa1a59-da82-416c-904c-61c8c9a8b4f6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.491350 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.491182    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/17fa1a59-da82-416c-904c-61c8c9a8b4f6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.491350 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.491221    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.491350 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.491253    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.491350 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.491289    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/17fa1a59-da82-416c-904c-61c8c9a8b4f6-config-out\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.491350 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.491331    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-web-config\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.491777 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.491754    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/17fa1a59-da82-416c-904c-61c8c9a8b4f6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.491907 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.491881    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.491971 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.491937    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17fa1a59-da82-416c-904c-61c8c9a8b4f6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.492034 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.491976    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.492034 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.492007    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.492137 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.492033    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/17fa1a59-da82-416c-904c-61c8c9a8b4f6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.492137 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.492071    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17fa1a59-da82-416c-904c-61c8c9a8b4f6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.492137 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.492100    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17fa1a59-da82-416c-904c-61c8c9a8b4f6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.492137 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.492130    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/17fa1a59-da82-416c-904c-61c8c9a8b4f6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.492322 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.492164    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.492322 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.492203    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/17fa1a59-da82-416c-904c-61c8c9a8b4f6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.492322 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.492231    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.492322 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.492265    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kts78\" (UniqueName: \"kubernetes.io/projected/17fa1a59-da82-416c-904c-61c8c9a8b4f6-kube-api-access-kts78\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.492322 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.492280    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17fa1a59-da82-416c-904c-61c8c9a8b4f6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.492622 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.492601    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/17fa1a59-da82-416c-904c-61c8c9a8b4f6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.492928 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.492899    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17fa1a59-da82-416c-904c-61c8c9a8b4f6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.493549 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.493523    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17fa1a59-da82-416c-904c-61c8c9a8b4f6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.494341 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.494112    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/17fa1a59-da82-416c-904c-61c8c9a8b4f6-config-out\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.494341 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.494211    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-web-config\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.494341 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.494228    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.494341 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.494253    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-config\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.494341 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.494269    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/17fa1a59-da82-416c-904c-61c8c9a8b4f6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.495510 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.494950    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17fa1a59-da82-416c-904c-61c8c9a8b4f6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.495510 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.495027    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.495510 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.495470    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/17fa1a59-da82-416c-904c-61c8c9a8b4f6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.495510 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.495508    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.496017 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.495996    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.496791 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.496770    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.496923 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.496905    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.496963 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.496914    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/17fa1a59-da82-416c-904c-61c8c9a8b4f6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.497405 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.497388    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/17fa1a59-da82-416c-904c-61c8c9a8b4f6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.504088 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.504069    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kts78\" (UniqueName: \"kubernetes.io/projected/17fa1a59-da82-416c-904c-61c8c9a8b4f6-kube-api-access-kts78\") pod \"prometheus-k8s-0\" (UID: \"17fa1a59-da82-416c-904c-61c8c9a8b4f6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.546184 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.546147    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:24:39.683736 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.683701    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 19:24:39.685626 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:24:39.685600    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17fa1a59_da82_416c_904c_61c8c9a8b4f6.slice/crio-8539010b009d24a74e3226731d1934febaf2cc60cec46b852df690d7af6f2d22 WatchSource:0}: Error finding container 8539010b009d24a74e3226731d1934febaf2cc60cec46b852df690d7af6f2d22: Status 404 returned error can't find the container with id 8539010b009d24a74e3226731d1934febaf2cc60cec46b852df690d7af6f2d22
Jul 13 19:24:39.883070 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.883033    2573 generic.go:356] "Generic (PLEG): container finished" podID="17fa1a59-da82-416c-904c-61c8c9a8b4f6" containerID="4817d33017ea628bab557af130720cbb69d0943f65d64aa5799f4fcfd7c7ff41" exitCode=0
Jul 13 19:24:39.883433 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.883117    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17fa1a59-da82-416c-904c-61c8c9a8b4f6","Type":"ContainerDied","Data":"4817d33017ea628bab557af130720cbb69d0943f65d64aa5799f4fcfd7c7ff41"}
Jul 13 19:24:39.883433 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:39.883180    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17fa1a59-da82-416c-904c-61c8c9a8b4f6","Type":"ContainerStarted","Data":"8539010b009d24a74e3226731d1934febaf2cc60cec46b852df690d7af6f2d22"}
Jul 13 19:24:40.891021 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:40.890982    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17fa1a59-da82-416c-904c-61c8c9a8b4f6","Type":"ContainerStarted","Data":"d9a31fe451995766c17b481d502a274c978d4963cca1659bf1e82f71327982dd"}
Jul 13 19:24:40.891462 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:40.891062    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17fa1a59-da82-416c-904c-61c8c9a8b4f6","Type":"ContainerStarted","Data":"d2224e255c43c979a5cac62f850289e820e186822ca9467847432420032fdbd1"}
Jul 13 19:24:40.891462 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:40.891100    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17fa1a59-da82-416c-904c-61c8c9a8b4f6","Type":"ContainerStarted","Data":"f13e822ceb02958bf10d11f8568c033df6300e1e9f236db9e3704dbfd679b18d"}
Jul 13 19:24:40.891462 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:40.891139    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17fa1a59-da82-416c-904c-61c8c9a8b4f6","Type":"ContainerStarted","Data":"5bc1367dadd053130337762a0bc9de01ff634274fa36844221c3e88392c7d71f"}
Jul 13 19:24:40.891462 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:40.891197    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17fa1a59-da82-416c-904c-61c8c9a8b4f6","Type":"ContainerStarted","Data":"d9583c5685ceff2208fce535b1386b858322371b502a405a4fb3c1c57fdb459e"}
Jul 13 19:24:40.891462 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:40.891234    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17fa1a59-da82-416c-904c-61c8c9a8b4f6","Type":"ContainerStarted","Data":"f11acd7ad5fd320ff1d0d5bb3ca255c8b9ad2aefd0cd767350476c3d1bd4deb8"}
Jul 13 19:24:40.922310 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:40.922187    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.902748555 podStartE2EDuration="1.922171763s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.423208ms" isStatefulPod=false podCreationTimestamp="2026-07-13 19:24:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:24:40.921056103 +0000 UTC m=+178.434932752" watchObservedRunningTime="2026-07-13 19:24:40.922171763 +0000 UTC m=+178.436048368"
Jul 13 19:24:41.048141 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:41.048096    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5373ef4e-030e-4489-8d3a-34bebd342e5a" path="/var/lib/kubelet/pods/5373ef4e-030e-4489-8d3a-34bebd342e5a/volumes"
Jul 13 19:24:44.547033 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:24:44.546991    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:25:39.546801 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:25:39.546755    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:25:39.562255 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:25:39.562230    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:25:40.131792 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:25:40.131761    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 19:26:42.854667 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:26:42.854622    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 19:26:42.857370 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:26:42.857340    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 19:26:42.859128 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:26:42.859109    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 19:26:42.861957 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:26:42.861926    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 19:26:42.865197 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:26:42.865174    2573 kubelet.go:1648] "Image garbage collection succeeded"
Jul 13 19:28:08.754302 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.754214    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv"]
Jul 13 19:28:08.756849 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.756827    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv"
Jul 13 19:28:08.759540 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.759519    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 13 19:28:08.759799 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.759779    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 13 19:28:08.760288 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.760269    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 13 19:28:08.760391 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.760335    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 13 19:28:08.760958 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.760940    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7pkj5\""
Jul 13 19:28:08.783329 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.783267    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv"]
Jul 13 19:28:08.836448 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.836415    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/350b25d0-f70f-4ff6-97df-1b1325c4bfed-webhook-cert\") pod \"opendatahub-operator-controller-manager-6996d85cfb-6mstv\" (UID: \"350b25d0-f70f-4ff6-97df-1b1325c4bfed\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv"
Jul 13 19:28:08.836448 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.836451    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phr2h\" (UniqueName: \"kubernetes.io/projected/350b25d0-f70f-4ff6-97df-1b1325c4bfed-kube-api-access-phr2h\") pod \"opendatahub-operator-controller-manager-6996d85cfb-6mstv\" (UID: \"350b25d0-f70f-4ff6-97df-1b1325c4bfed\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv"
Jul 13 19:28:08.836676 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.836474    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/350b25d0-f70f-4ff6-97df-1b1325c4bfed-tmp\") pod \"opendatahub-operator-controller-manager-6996d85cfb-6mstv\" (UID: \"350b25d0-f70f-4ff6-97df-1b1325c4bfed\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv"
Jul 13 19:28:08.836676 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.836555    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/350b25d0-f70f-4ff6-97df-1b1325c4bfed-manifests\") pod \"opendatahub-operator-controller-manager-6996d85cfb-6mstv\" (UID: \"350b25d0-f70f-4ff6-97df-1b1325c4bfed\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv"
Jul 13 19:28:08.836676 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.836583    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/350b25d0-f70f-4ff6-97df-1b1325c4bfed-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6996d85cfb-6mstv\" (UID: \"350b25d0-f70f-4ff6-97df-1b1325c4bfed\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv"
Jul 13 19:28:08.937041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.937001    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/350b25d0-f70f-4ff6-97df-1b1325c4bfed-manifests\") pod \"opendatahub-operator-controller-manager-6996d85cfb-6mstv\" (UID: \"350b25d0-f70f-4ff6-97df-1b1325c4bfed\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv"
Jul 13 19:28:08.937041 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.937040    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/350b25d0-f70f-4ff6-97df-1b1325c4bfed-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6996d85cfb-6mstv\" (UID: \"350b25d0-f70f-4ff6-97df-1b1325c4bfed\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv"
Jul 13 19:28:08.937244 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.937091    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/350b25d0-f70f-4ff6-97df-1b1325c4bfed-webhook-cert\") pod \"opendatahub-operator-controller-manager-6996d85cfb-6mstv\" (UID: \"350b25d0-f70f-4ff6-97df-1b1325c4bfed\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv"
Jul 13 19:28:08.937244 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.937113    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phr2h\" (UniqueName: \"kubernetes.io/projected/350b25d0-f70f-4ff6-97df-1b1325c4bfed-kube-api-access-phr2h\") pod \"opendatahub-operator-controller-manager-6996d85cfb-6mstv\" (UID: \"350b25d0-f70f-4ff6-97df-1b1325c4bfed\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv"
Jul 13 19:28:08.937244 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.937144    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/350b25d0-f70f-4ff6-97df-1b1325c4bfed-tmp\") pod \"opendatahub-operator-controller-manager-6996d85cfb-6mstv\" (UID: \"350b25d0-f70f-4ff6-97df-1b1325c4bfed\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv"
Jul 13 19:28:08.937457 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.937433    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/350b25d0-f70f-4ff6-97df-1b1325c4bfed-manifests\") pod \"opendatahub-operator-controller-manager-6996d85cfb-6mstv\" (UID: \"350b25d0-f70f-4ff6-97df-1b1325c4bfed\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv"
Jul 13 19:28:08.939501 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.939468    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/350b25d0-f70f-4ff6-97df-1b1325c4bfed-tmp\") pod \"opendatahub-operator-controller-manager-6996d85cfb-6mstv\" (UID: \"350b25d0-f70f-4ff6-97df-1b1325c4bfed\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv"
Jul 13 19:28:08.939660 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.939585    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/350b25d0-f70f-4ff6-97df-1b1325c4bfed-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6996d85cfb-6mstv\" (UID: \"350b25d0-f70f-4ff6-97df-1b1325c4bfed\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv"
Jul 13 19:28:08.939660 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.939597    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/350b25d0-f70f-4ff6-97df-1b1325c4bfed-webhook-cert\") pod \"opendatahub-operator-controller-manager-6996d85cfb-6mstv\" (UID: \"350b25d0-f70f-4ff6-97df-1b1325c4bfed\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv"
Jul 13 19:28:08.946090 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:08.946062    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phr2h\" (UniqueName: \"kubernetes.io/projected/350b25d0-f70f-4ff6-97df-1b1325c4bfed-kube-api-access-phr2h\") pod \"opendatahub-operator-controller-manager-6996d85cfb-6mstv\" (UID: \"350b25d0-f70f-4ff6-97df-1b1325c4bfed\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv"
Jul 13 19:28:09.067140 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:09.067046    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv"
Jul 13 19:28:09.204155 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:09.204114    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv"]
Jul 13 19:28:09.206526 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:28:09.206494    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod350b25d0_f70f_4ff6_97df_1b1325c4bfed.slice/crio-f861842e9992da9882f2015f2b1b03092f1ecf96517a7c136e473a1446007d66 WatchSource:0}: Error finding container f861842e9992da9882f2015f2b1b03092f1ecf96517a7c136e473a1446007d66: Status 404 returned error can't find the container with id f861842e9992da9882f2015f2b1b03092f1ecf96517a7c136e473a1446007d66
Jul 13 19:28:09.208766 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:09.208741    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 19:28:09.687673 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:09.687597    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv" event={"ID":"350b25d0-f70f-4ff6-97df-1b1325c4bfed","Type":"ContainerStarted","Data":"f861842e9992da9882f2015f2b1b03092f1ecf96517a7c136e473a1446007d66"}
Jul 13 19:28:12.703097 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:12.703062    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv" event={"ID":"350b25d0-f70f-4ff6-97df-1b1325c4bfed","Type":"ContainerStarted","Data":"7717e15d1cce99266bc7e5130556254ebb1dfd6d3516aac8059b6cb62eae173b"}
Jul 13 19:28:13.709255 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:13.709226    2573 generic.go:356] "Generic (PLEG): container finished" podID="350b25d0-f70f-4ff6-97df-1b1325c4bfed" containerID="7717e15d1cce99266bc7e5130556254ebb1dfd6d3516aac8059b6cb62eae173b" exitCode=0
Jul 13 19:28:13.709663 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:13.709273    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv" event={"ID":"350b25d0-f70f-4ff6-97df-1b1325c4bfed","Type":"ContainerDied","Data":"7717e15d1cce99266bc7e5130556254ebb1dfd6d3516aac8059b6cb62eae173b"}
Jul 13 19:28:14.716887 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:14.716844    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv" event={"ID":"350b25d0-f70f-4ff6-97df-1b1325c4bfed","Type":"ContainerStarted","Data":"524adaa5e95c6e66791f68ac7775370fa4a970912d3aa80fb99c9059905378d5"}
Jul 13 19:28:14.717288 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:14.716938    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv"
Jul 13 19:28:14.739752 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:14.739695    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv" podStartSLOduration=1.974934363 podStartE2EDuration="6.739671688s" totalImagesPullingTime="3.742035816s" totalInitContainerRuntime="1.022701509s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:28:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:28:14.737920642 +0000 UTC m=+392.251797269" watchObservedRunningTime="2026-07-13 19:28:14.739671688 +0000 UTC m=+392.253548298"
Jul 13 19:28:25.723961 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:25.723930    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-6mstv"
Jul 13 19:28:29.850799 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:29.850744    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-55f6586c87-gbf8l"]
Jul 13 19:28:29.866406 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:29.866375    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-55f6586c87-gbf8l"
Jul 13 19:28:29.870031 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:29.869728    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 13 19:28:29.870434 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:29.870386    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-87dwb\""
Jul 13 19:28:29.870844 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:29.870808    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 13 19:28:29.871912 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:29.871886    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-55f6586c87-gbf8l"]
Jul 13 19:28:29.901521 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:29.901490    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsxrd\" (UniqueName: \"kubernetes.io/projected/6a67ca80-5190-41d6-96d0-5bdc1047fc6f-kube-api-access-gsxrd\") pod \"kube-auth-proxy-55f6586c87-gbf8l\" (UID: \"6a67ca80-5190-41d6-96d0-5bdc1047fc6f\") " pod="openshift-ingress/kube-auth-proxy-55f6586c87-gbf8l"
Jul 13 19:28:29.901713 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:29.901531    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a67ca80-5190-41d6-96d0-5bdc1047fc6f-tmp\") pod \"kube-auth-proxy-55f6586c87-gbf8l\" (UID: \"6a67ca80-5190-41d6-96d0-5bdc1047fc6f\") " pod="openshift-ingress/kube-auth-proxy-55f6586c87-gbf8l"
Jul 13 19:28:29.901713 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:29.901579    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a67ca80-5190-41d6-96d0-5bdc1047fc6f-tls-certs\") pod \"kube-auth-proxy-55f6586c87-gbf8l\" (UID: \"6a67ca80-5190-41d6-96d0-5bdc1047fc6f\") " pod="openshift-ingress/kube-auth-proxy-55f6586c87-gbf8l"
Jul 13 19:28:30.002894 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:30.002855    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsxrd\" (UniqueName: \"kubernetes.io/projected/6a67ca80-5190-41d6-96d0-5bdc1047fc6f-kube-api-access-gsxrd\") pod \"kube-auth-proxy-55f6586c87-gbf8l\" (UID: \"6a67ca80-5190-41d6-96d0-5bdc1047fc6f\") " pod="openshift-ingress/kube-auth-proxy-55f6586c87-gbf8l"
Jul 13 19:28:30.003087 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:30.002902    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a67ca80-5190-41d6-96d0-5bdc1047fc6f-tmp\") pod \"kube-auth-proxy-55f6586c87-gbf8l\" (UID: \"6a67ca80-5190-41d6-96d0-5bdc1047fc6f\") " pod="openshift-ingress/kube-auth-proxy-55f6586c87-gbf8l"
Jul 13 19:28:30.003087 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:30.002955    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a67ca80-5190-41d6-96d0-5bdc1047fc6f-tls-certs\") pod \"kube-auth-proxy-55f6586c87-gbf8l\" (UID: \"6a67ca80-5190-41d6-96d0-5bdc1047fc6f\") " pod="openshift-ingress/kube-auth-proxy-55f6586c87-gbf8l"
Jul 13 19:28:30.005308 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:30.005280    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a67ca80-5190-41d6-96d0-5bdc1047fc6f-tmp\") pod \"kube-auth-proxy-55f6586c87-gbf8l\" (UID: \"6a67ca80-5190-41d6-96d0-5bdc1047fc6f\") " pod="openshift-ingress/kube-auth-proxy-55f6586c87-gbf8l"
Jul 13 19:28:30.005423 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:30.005395    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a67ca80-5190-41d6-96d0-5bdc1047fc6f-tls-certs\") pod \"kube-auth-proxy-55f6586c87-gbf8l\" (UID: \"6a67ca80-5190-41d6-96d0-5bdc1047fc6f\") " pod="openshift-ingress/kube-auth-proxy-55f6586c87-gbf8l"
Jul 13 19:28:30.019316 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:30.019286    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsxrd\" (UniqueName: \"kubernetes.io/projected/6a67ca80-5190-41d6-96d0-5bdc1047fc6f-kube-api-access-gsxrd\") pod \"kube-auth-proxy-55f6586c87-gbf8l\" (UID: \"6a67ca80-5190-41d6-96d0-5bdc1047fc6f\") " pod="openshift-ingress/kube-auth-proxy-55f6586c87-gbf8l"
Jul 13 19:28:30.064677 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:30.064629    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-k9wqz"]
Jul 13 19:28:30.068903 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:30.068875    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-k9wqz"
Jul 13 19:28:30.072100 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:30.072078    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 13 19:28:30.073483 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:30.073462    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-6j8qg\""
Jul 13 19:28:30.081570 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:30.081549    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-k9wqz"]
Jul 13 19:28:30.104094 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:30.104000    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l7dj\" (UniqueName: \"kubernetes.io/projected/d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75-kube-api-access-5l7dj\") pod \"odh-model-controller-7998c67dc4-k9wqz\" (UID: \"d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75\") " pod="opendatahub/odh-model-controller-7998c67dc4-k9wqz"
Jul 13 19:28:30.104094 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:30.104070    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75-cert\") pod \"odh-model-controller-7998c67dc4-k9wqz\" (UID: \"d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75\") " pod="opendatahub/odh-model-controller-7998c67dc4-k9wqz"
Jul 13 19:28:30.178368 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:30.178331    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-55f6586c87-gbf8l"
Jul 13 19:28:30.204734 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:30.204695    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l7dj\" (UniqueName: \"kubernetes.io/projected/d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75-kube-api-access-5l7dj\") pod \"odh-model-controller-7998c67dc4-k9wqz\" (UID: \"d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75\") " pod="opendatahub/odh-model-controller-7998c67dc4-k9wqz"
Jul 13 19:28:30.204910 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:30.204773    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75-cert\") pod \"odh-model-controller-7998c67dc4-k9wqz\" (UID: \"d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75\") " pod="opendatahub/odh-model-controller-7998c67dc4-k9wqz"
Jul 13 19:28:30.204910 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:28:30.204899    2573 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 13 19:28:30.205031 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:28:30.204978    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75-cert podName:d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75 nodeName:}" failed. No retries permitted until 2026-07-13 19:28:30.704955987 +0000 UTC m=+408.218832572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75-cert") pod "odh-model-controller-7998c67dc4-k9wqz" (UID: "d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75") : secret "odh-model-controller-webhook-cert" not found
Jul 13 19:28:30.220371 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:30.220344    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l7dj\" (UniqueName: \"kubernetes.io/projected/d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75-kube-api-access-5l7dj\") pod \"odh-model-controller-7998c67dc4-k9wqz\" (UID: \"d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75\") " pod="opendatahub/odh-model-controller-7998c67dc4-k9wqz"
Jul 13 19:28:30.320197 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:30.320167    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-55f6586c87-gbf8l"]
Jul 13 19:28:30.322602 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:28:30.322574    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a67ca80_5190_41d6_96d0_5bdc1047fc6f.slice/crio-98416e6acfc0b6c6470d6fa53341e5d46f7dfbdd059c0ad3e4c61f9d17ccf7ce WatchSource:0}: Error finding container 98416e6acfc0b6c6470d6fa53341e5d46f7dfbdd059c0ad3e4c61f9d17ccf7ce: Status 404 returned error can't find the container with id 98416e6acfc0b6c6470d6fa53341e5d46f7dfbdd059c0ad3e4c61f9d17ccf7ce
Jul 13 19:28:30.709010 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:30.708971    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75-cert\") pod \"odh-model-controller-7998c67dc4-k9wqz\" (UID: \"d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75\") " pod="opendatahub/odh-model-controller-7998c67dc4-k9wqz"
Jul 13 19:28:30.709194 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:28:30.709133    2573 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 13 19:28:30.709236 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:28:30.709194    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75-cert podName:d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75 nodeName:}" failed. No retries permitted until 2026-07-13 19:28:31.709176336 +0000 UTC m=+409.223052921 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75-cert") pod "odh-model-controller-7998c67dc4-k9wqz" (UID: "d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75") : secret "odh-model-controller-webhook-cert" not found
Jul 13 19:28:30.786451 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:30.786418    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-55f6586c87-gbf8l" event={"ID":"6a67ca80-5190-41d6-96d0-5bdc1047fc6f","Type":"ContainerStarted","Data":"98416e6acfc0b6c6470d6fa53341e5d46f7dfbdd059c0ad3e4c61f9d17ccf7ce"}
Jul 13 19:28:31.616563 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.616510    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"]
Jul 13 19:28:31.620121 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.620098    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:31.623224 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.623200    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 13 19:28:31.623387 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.623344    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 13 19:28:31.623464 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.623428    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 13 19:28:31.623521 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.623472    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-d2fvc\""
Jul 13 19:28:31.623783 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.623768    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 13 19:28:31.650263 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.650223    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"]
Jul 13 19:28:31.719039 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.719005    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/71f388ab-9def-4bbb-a25a-f46465266c4b-local-certs\") pod \"istiod-openshift-gateway-7574459656-5pf8c\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:31.719217 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.719047    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5pf8c\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:31.719217 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.719152    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-token\") pod \"istiod-openshift-gateway-7574459656-5pf8c\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:31.719288 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.719217    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2vfd\" (UniqueName: \"kubernetes.io/projected/71f388ab-9def-4bbb-a25a-f46465266c4b-kube-api-access-d2vfd\") pod \"istiod-openshift-gateway-7574459656-5pf8c\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:31.719288 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.719256    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5pf8c\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:31.719353 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.719316    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5pf8c\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:31.719394 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.719352    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75-cert\") pod \"odh-model-controller-7998c67dc4-k9wqz\" (UID: \"d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75\") " pod="opendatahub/odh-model-controller-7998c67dc4-k9wqz"
Jul 13 19:28:31.719394 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.719389    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/71f388ab-9def-4bbb-a25a-f46465266c4b-cacerts\") pod \"istiod-openshift-gateway-7574459656-5pf8c\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:31.721910 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.721884    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75-cert\") pod \"odh-model-controller-7998c67dc4-k9wqz\" (UID: \"d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75\") " pod="opendatahub/odh-model-controller-7998c67dc4-k9wqz"
Jul 13 19:28:31.820874 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.820835    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2vfd\" (UniqueName: \"kubernetes.io/projected/71f388ab-9def-4bbb-a25a-f46465266c4b-kube-api-access-d2vfd\") pod \"istiod-openshift-gateway-7574459656-5pf8c\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:31.820874 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.820889    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5pf8c\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:31.821136 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.820931    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5pf8c\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:31.821136 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.820971    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/71f388ab-9def-4bbb-a25a-f46465266c4b-cacerts\") pod \"istiod-openshift-gateway-7574459656-5pf8c\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:31.821136 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.821024    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/71f388ab-9def-4bbb-a25a-f46465266c4b-local-certs\") pod \"istiod-openshift-gateway-7574459656-5pf8c\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:31.821286 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.821174    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5pf8c\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:31.821286 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.821241    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-token\") pod \"istiod-openshift-gateway-7574459656-5pf8c\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:31.821734 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.821613    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5pf8c\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:31.824051 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.823981    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/71f388ab-9def-4bbb-a25a-f46465266c4b-local-certs\") pod \"istiod-openshift-gateway-7574459656-5pf8c\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:31.824168 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.824151    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5pf8c\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:31.824286 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.824264    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5pf8c\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:31.824334 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.824271    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/71f388ab-9def-4bbb-a25a-f46465266c4b-cacerts\") pod \"istiod-openshift-gateway-7574459656-5pf8c\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:31.832608 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.832585    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2vfd\" (UniqueName: \"kubernetes.io/projected/71f388ab-9def-4bbb-a25a-f46465266c4b-kube-api-access-d2vfd\") pod \"istiod-openshift-gateway-7574459656-5pf8c\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:31.834726 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.834685    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-token\") pod \"istiod-openshift-gateway-7574459656-5pf8c\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:31.879660 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.879543    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-k9wqz"
Jul 13 19:28:31.932294 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:31.932254    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:32.069397 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:32.069249    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-k9wqz"]
Jul 13 19:28:32.072674 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:28:32.072608    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd092db6a_ddb6_46e4_ac4c_4c9f8e39ef75.slice/crio-84e15da1bfb4fda5c2afc02170b22c7a9bc70eb15fa0b01714201c5e99f2c9d3 WatchSource:0}: Error finding container 84e15da1bfb4fda5c2afc02170b22c7a9bc70eb15fa0b01714201c5e99f2c9d3: Status 404 returned error can't find the container with id 84e15da1bfb4fda5c2afc02170b22c7a9bc70eb15fa0b01714201c5e99f2c9d3
Jul 13 19:28:32.108299 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:32.108267    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"]
Jul 13 19:28:32.111318 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:28:32.111278    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71f388ab_9def_4bbb_a25a_f46465266c4b.slice/crio-aa6633b41adc09da6e2c938074b8ec4bc8391d637bd2945c10c04694a0f3c7fc WatchSource:0}: Error finding container aa6633b41adc09da6e2c938074b8ec4bc8391d637bd2945c10c04694a0f3c7fc: Status 404 returned error can't find the container with id aa6633b41adc09da6e2c938074b8ec4bc8391d637bd2945c10c04694a0f3c7fc
Jul 13 19:28:32.797497 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:32.797441    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-k9wqz" event={"ID":"d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75","Type":"ContainerStarted","Data":"84e15da1bfb4fda5c2afc02170b22c7a9bc70eb15fa0b01714201c5e99f2c9d3"}
Jul 13 19:28:32.798917 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:32.798881    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c" event={"ID":"71f388ab-9def-4bbb-a25a-f46465266c4b","Type":"ContainerStarted","Data":"aa6633b41adc09da6e2c938074b8ec4bc8391d637bd2945c10c04694a0f3c7fc"}
Jul 13 19:28:33.805556 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:33.805516    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-55f6586c87-gbf8l" event={"ID":"6a67ca80-5190-41d6-96d0-5bdc1047fc6f","Type":"ContainerStarted","Data":"fc1ce9802bf65accfae25508492b12e5692c13d31b8b072ecbcaf548f35fe03e"}
Jul 13 19:28:33.831039 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:33.830962    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-55f6586c87-gbf8l" podStartSLOduration=1.924560467 podStartE2EDuration="4.830941572s" totalImagesPullingTime="2.906381105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:28:33.830334299 +0000 UTC m=+411.344210926" watchObservedRunningTime="2026-07-13 19:28:33.830941572 +0000 UTC m=+411.344818249"
Jul 13 19:28:37.191116 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:37.191072    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 13 19:28:37.191405 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:37.191158    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 13 19:28:37.828731 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:37.828691    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c" event={"ID":"71f388ab-9def-4bbb-a25a-f46465266c4b","Type":"ContainerStarted","Data":"bc8bf02094ea8b984a2424e4a0bcff9dca78672b4ebc6a3f9be3a121c8d852b8"}
Jul 13 19:28:37.829046 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:37.828953    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:37.830520 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:37.830494    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-k9wqz" event={"ID":"d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75","Type":"ContainerStarted","Data":"1513a30698e505482164b0132d88843ecddf60f91e1bfc67911b1b8a5f541b4b"}
Jul 13 19:28:37.830671 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:37.830559    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-k9wqz"
Jul 13 19:28:37.854081 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:37.854020    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c" podStartSLOduration=1.77730467 podStartE2EDuration="6.853998376s" totalImagesPullingTime="5.076693706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:28:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:28:37.853267239 +0000 UTC m=+415.367143860" watchObservedRunningTime="2026-07-13 19:28:37.853998376 +0000 UTC m=+415.367874983"
Jul 13 19:28:37.877013 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:37.876950    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-k9wqz" podStartSLOduration=2.712487319 podStartE2EDuration="7.876930776s" totalImagesPullingTime="5.164443457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:28:37.875372164 +0000 UTC m=+415.389248795" watchObservedRunningTime="2026-07-13 19:28:37.876930776 +0000 UTC m=+415.390807383"
Jul 13 19:28:38.836438 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:38.836397    2573 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-5pf8c container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 13 19:28:38.836836 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:38.836452    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c" podUID="71f388ab-9def-4bbb-a25a-f46465266c4b" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:28:41.018536 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.018499    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-655465cc8d-kqlmd"]
Jul 13 19:28:41.022182 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.022152    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-655465cc8d-kqlmd"
Jul 13 19:28:41.027331 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.027304    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 13 19:28:41.027331 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.027322    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 13 19:28:41.027559 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.027403    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 13 19:28:41.027559 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.027413    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 13 19:28:41.027559 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.027425    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2qgqg\""
Jul 13 19:28:41.027559 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.027407    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 13 19:28:41.048445 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.048416    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-655465cc8d-kqlmd"]
Jul 13 19:28:41.109406 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.109366    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d-metrics-cert\") pod \"lws-controller-manager-655465cc8d-kqlmd\" (UID: \"d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-kqlmd"
Jul 13 19:28:41.109728 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.109702    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d-manager-config\") pod \"lws-controller-manager-655465cc8d-kqlmd\" (UID: \"d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-kqlmd"
Jul 13 19:28:41.109853 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.109837    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7f48\" (UniqueName: \"kubernetes.io/projected/d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d-kube-api-access-q7f48\") pod \"lws-controller-manager-655465cc8d-kqlmd\" (UID: \"d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-kqlmd"
Jul 13 19:28:41.110014 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.109999    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d-cert\") pod \"lws-controller-manager-655465cc8d-kqlmd\" (UID: \"d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-kqlmd"
Jul 13 19:28:41.210757 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.210715    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d-metrics-cert\") pod \"lws-controller-manager-655465cc8d-kqlmd\" (UID: \"d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-kqlmd"
Jul 13 19:28:41.210946 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.210779    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d-manager-config\") pod \"lws-controller-manager-655465cc8d-kqlmd\" (UID: \"d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-kqlmd"
Jul 13 19:28:41.210946 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.210800    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7f48\" (UniqueName: \"kubernetes.io/projected/d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d-kube-api-access-q7f48\") pod \"lws-controller-manager-655465cc8d-kqlmd\" (UID: \"d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-kqlmd"
Jul 13 19:28:41.210946 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.210845    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d-cert\") pod \"lws-controller-manager-655465cc8d-kqlmd\" (UID: \"d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-kqlmd"
Jul 13 19:28:41.211618 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.211589    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d-manager-config\") pod \"lws-controller-manager-655465cc8d-kqlmd\" (UID: \"d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-kqlmd"
Jul 13 19:28:41.213421 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.213396    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d-metrics-cert\") pod \"lws-controller-manager-655465cc8d-kqlmd\" (UID: \"d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-kqlmd"
Jul 13 19:28:41.213527 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.213502    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d-cert\") pod \"lws-controller-manager-655465cc8d-kqlmd\" (UID: \"d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-kqlmd"
Jul 13 19:28:41.221271 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.221242    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7f48\" (UniqueName: \"kubernetes.io/projected/d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d-kube-api-access-q7f48\") pod \"lws-controller-manager-655465cc8d-kqlmd\" (UID: \"d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-kqlmd"
Jul 13 19:28:41.332620 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.332522    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-655465cc8d-kqlmd"
Jul 13 19:28:41.510764 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.510565    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-655465cc8d-kqlmd"]
Jul 13 19:28:41.513361 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:28:41.513324    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd98a29a9_caf1_4db4_a7d3_bfb08bc4a40d.slice/crio-bdf735fcd19f6dae819cf53b51ca62e6b1164eb384ae00420d9f9484b2a99861 WatchSource:0}: Error finding container bdf735fcd19f6dae819cf53b51ca62e6b1164eb384ae00420d9f9484b2a99861: Status 404 returned error can't find the container with id bdf735fcd19f6dae819cf53b51ca62e6b1164eb384ae00420d9f9484b2a99861
Jul 13 19:28:41.835555 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.835518    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:41.848956 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:41.848924    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-655465cc8d-kqlmd" event={"ID":"d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d","Type":"ContainerStarted","Data":"bdf735fcd19f6dae819cf53b51ca62e6b1164eb384ae00420d9f9484b2a99861"}
Jul 13 19:28:43.570420 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:43.570377    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"]
Jul 13 19:28:43.570923 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:43.570622    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c" podUID="71f388ab-9def-4bbb-a25a-f46465266c4b" containerName="discovery" containerID="cri-o://bc8bf02094ea8b984a2424e4a0bcff9dca78672b4ebc6a3f9be3a121c8d852b8" gracePeriod=30
Jul 13 19:28:43.865172 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:43.865140    2573 generic.go:356] "Generic (PLEG): container finished" podID="71f388ab-9def-4bbb-a25a-f46465266c4b" containerID="bc8bf02094ea8b984a2424e4a0bcff9dca78672b4ebc6a3f9be3a121c8d852b8" exitCode=0
Jul 13 19:28:43.865324 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:43.865181    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c" event={"ID":"71f388ab-9def-4bbb-a25a-f46465266c4b","Type":"ContainerDied","Data":"bc8bf02094ea8b984a2424e4a0bcff9dca78672b4ebc6a3f9be3a121c8d852b8"}
Jul 13 19:28:43.923399 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:43.923373    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:44.040895 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.040856    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/71f388ab-9def-4bbb-a25a-f46465266c4b-local-certs\") pod \"71f388ab-9def-4bbb-a25a-f46465266c4b\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") "
Jul 13 19:28:44.041094 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.040909    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-token\") pod \"71f388ab-9def-4bbb-a25a-f46465266c4b\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") "
Jul 13 19:28:44.041094 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.040936    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-csr-dns-cert\") pod \"71f388ab-9def-4bbb-a25a-f46465266c4b\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") "
Jul 13 19:28:44.041094 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.040975    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/71f388ab-9def-4bbb-a25a-f46465266c4b-cacerts\") pod \"71f388ab-9def-4bbb-a25a-f46465266c4b\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") "
Jul 13 19:28:44.041094 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.041002    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-kubeconfig\") pod \"71f388ab-9def-4bbb-a25a-f46465266c4b\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") "
Jul 13 19:28:44.041094 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.041035    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-csr-ca-configmap\") pod \"71f388ab-9def-4bbb-a25a-f46465266c4b\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") "
Jul 13 19:28:44.041094 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.041071    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2vfd\" (UniqueName: \"kubernetes.io/projected/71f388ab-9def-4bbb-a25a-f46465266c4b-kube-api-access-d2vfd\") pod \"71f388ab-9def-4bbb-a25a-f46465266c4b\" (UID: \"71f388ab-9def-4bbb-a25a-f46465266c4b\") "
Jul 13 19:28:44.041569 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.041509    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "71f388ab-9def-4bbb-a25a-f46465266c4b" (UID: "71f388ab-9def-4bbb-a25a-f46465266c4b"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:28:44.043354 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.043322    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71f388ab-9def-4bbb-a25a-f46465266c4b-local-certs" (OuterVolumeSpecName: "local-certs") pod "71f388ab-9def-4bbb-a25a-f46465266c4b" (UID: "71f388ab-9def-4bbb-a25a-f46465266c4b"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:28:44.043461 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.043416    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71f388ab-9def-4bbb-a25a-f46465266c4b-kube-api-access-d2vfd" (OuterVolumeSpecName: "kube-api-access-d2vfd") pod "71f388ab-9def-4bbb-a25a-f46465266c4b" (UID: "71f388ab-9def-4bbb-a25a-f46465266c4b"). InnerVolumeSpecName "kube-api-access-d2vfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:28:44.043555 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.043538    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-token" (OuterVolumeSpecName: "istio-token") pod "71f388ab-9def-4bbb-a25a-f46465266c4b" (UID: "71f388ab-9def-4bbb-a25a-f46465266c4b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:28:44.043603 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.043565    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71f388ab-9def-4bbb-a25a-f46465266c4b-cacerts" (OuterVolumeSpecName: "cacerts") pod "71f388ab-9def-4bbb-a25a-f46465266c4b" (UID: "71f388ab-9def-4bbb-a25a-f46465266c4b"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:28:44.043670 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.043622    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "71f388ab-9def-4bbb-a25a-f46465266c4b" (UID: "71f388ab-9def-4bbb-a25a-f46465266c4b"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:28:44.043710 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.043651    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "71f388ab-9def-4bbb-a25a-f46465266c4b" (UID: "71f388ab-9def-4bbb-a25a-f46465266c4b"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:28:44.142394 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.142313    2573 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/71f388ab-9def-4bbb-a25a-f46465266c4b-cacerts\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:28:44.142394 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.142342    2573 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-kubeconfig\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:28:44.142394 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.142354    2573 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-csr-ca-configmap\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:28:44.142394 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.142362    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2vfd\" (UniqueName: \"kubernetes.io/projected/71f388ab-9def-4bbb-a25a-f46465266c4b-kube-api-access-d2vfd\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:28:44.142394 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.142371    2573 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/71f388ab-9def-4bbb-a25a-f46465266c4b-local-certs\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:28:44.142394 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.142379    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-token\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:28:44.142394 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.142388    2573 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/71f388ab-9def-4bbb-a25a-f46465266c4b-istio-csr-dns-cert\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:28:44.870902 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.870869    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"
Jul 13 19:28:44.871350 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.870869    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c" event={"ID":"71f388ab-9def-4bbb-a25a-f46465266c4b","Type":"ContainerDied","Data":"aa6633b41adc09da6e2c938074b8ec4bc8391d637bd2945c10c04694a0f3c7fc"}
Jul 13 19:28:44.871350 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.871003    2573 scope.go:117] "RemoveContainer" containerID="bc8bf02094ea8b984a2424e4a0bcff9dca78672b4ebc6a3f9be3a121c8d852b8"
Jul 13 19:28:44.872728 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.872671    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-655465cc8d-kqlmd" event={"ID":"d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d","Type":"ContainerStarted","Data":"39369d175ab2a9ef274369d416de62ae6b0d3cec7f71e63c0504f9f6cd346371"}
Jul 13 19:28:44.872816 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.872753    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-655465cc8d-kqlmd"
Jul 13 19:28:44.895818 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.895765    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-655465cc8d-kqlmd" podStartSLOduration=2.572422155 podStartE2EDuration="4.895747925s" totalImagesPullingTime="2.32332577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:28:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:28:44.893591235 +0000 UTC m=+422.407467842" watchObservedRunningTime="2026-07-13 19:28:44.895747925 +0000 UTC m=+422.409624531"
Jul 13 19:28:44.911605 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.911573    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"]
Jul 13 19:28:44.920153 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:44.920127    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5pf8c"]
Jul 13 19:28:45.048760 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:45.048687    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71f388ab-9def-4bbb-a25a-f46465266c4b" path="/var/lib/kubelet/pods/71f388ab-9def-4bbb-a25a-f46465266c4b/volumes"
Jul 13 19:28:48.837422 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:48.837391    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-k9wqz"
Jul 13 19:28:55.880833 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:28:55.880794    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-655465cc8d-kqlmd"
Jul 13 19:30:18.849428 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:18.849393    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-pxkvw"]
Jul 13 19:30:18.849922 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:18.849772    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71f388ab-9def-4bbb-a25a-f46465266c4b" containerName="discovery"
Jul 13 19:30:18.849922 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:18.849786    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="71f388ab-9def-4bbb-a25a-f46465266c4b" containerName="discovery"
Jul 13 19:30:18.849922 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:18.849842    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="71f388ab-9def-4bbb-a25a-f46465266c4b" containerName="discovery"
Jul 13 19:30:18.851748 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:18.851726    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pxkvw"
Jul 13 19:30:18.854018 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:18.853997    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-phzkx\""
Jul 13 19:30:18.856508 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:18.856486    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 13 19:30:18.856741 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:18.856724    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 13 19:30:18.864661 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:18.864615    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-pxkvw"]
Jul 13 19:30:18.940582 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:18.940549    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2pvg\" (UniqueName: \"kubernetes.io/projected/e341a781-5754-4b8b-ab7d-a09308d95354-kube-api-access-p2pvg\") pod \"authorino-operator-75cc8d5cb7-pxkvw\" (UID: \"e341a781-5754-4b8b-ab7d-a09308d95354\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-pxkvw"
Jul 13 19:30:19.041024 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:19.040987    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2pvg\" (UniqueName: \"kubernetes.io/projected/e341a781-5754-4b8b-ab7d-a09308d95354-kube-api-access-p2pvg\") pod \"authorino-operator-75cc8d5cb7-pxkvw\" (UID: \"e341a781-5754-4b8b-ab7d-a09308d95354\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-pxkvw"
Jul 13 19:30:19.055178 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:19.055145    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2pvg\" (UniqueName: \"kubernetes.io/projected/e341a781-5754-4b8b-ab7d-a09308d95354-kube-api-access-p2pvg\") pod \"authorino-operator-75cc8d5cb7-pxkvw\" (UID: \"e341a781-5754-4b8b-ab7d-a09308d95354\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-pxkvw"
Jul 13 19:30:19.162036 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:19.161940    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pxkvw"
Jul 13 19:30:19.307862 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:19.307833    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-pxkvw"]
Jul 13 19:30:19.310237 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:30:19.310208    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode341a781_5754_4b8b_ab7d_a09308d95354.slice/crio-d4a3706e0829088ca6b25e86cfd0832f92411d45f9d244f7334e950363529574 WatchSource:0}: Error finding container d4a3706e0829088ca6b25e86cfd0832f92411d45f9d244f7334e950363529574: Status 404 returned error can't find the container with id d4a3706e0829088ca6b25e86cfd0832f92411d45f9d244f7334e950363529574
Jul 13 19:30:20.283245 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:20.283168    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pxkvw" event={"ID":"e341a781-5754-4b8b-ab7d-a09308d95354","Type":"ContainerStarted","Data":"d4a3706e0829088ca6b25e86cfd0832f92411d45f9d244f7334e950363529574"}
Jul 13 19:30:22.293712 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:22.293681    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pxkvw" event={"ID":"e341a781-5754-4b8b-ab7d-a09308d95354","Type":"ContainerStarted","Data":"05bac36219dde8b84c01ad4ab4e3d2654bead159a30590a65b1f9637b5621249"}
Jul 13 19:30:22.294109 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:22.293771    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pxkvw"
Jul 13 19:30:22.311446 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:22.311378    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pxkvw" podStartSLOduration=1.967617487 podStartE2EDuration="4.311357565s" totalImagesPullingTime="2.343740078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:30:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:30:22.310526429 +0000 UTC m=+519.824403034" watchObservedRunningTime="2026-07-13 19:30:22.311357565 +0000 UTC m=+519.825234171"
Jul 13 19:30:27.240549 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:27.240509    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-prjbj"]
Jul 13 19:30:27.243016 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:27.242991    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prjbj"
Jul 13 19:30:27.245727 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:27.245692    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 13 19:30:27.245727 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:27.245722    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 13 19:30:27.245929 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:27.245830    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-88br4\""
Jul 13 19:30:27.252695 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:27.252668    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-prjbj"]
Jul 13 19:30:27.315756 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:27.315727    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/40dbad3e-3816-4a6e-96ac-368f118495fe-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-prjbj\" (UID: \"40dbad3e-3816-4a6e-96ac-368f118495fe\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prjbj"
Jul 13 19:30:27.315756 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:27.315760    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/40dbad3e-3816-4a6e-96ac-368f118495fe-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-prjbj\" (UID: \"40dbad3e-3816-4a6e-96ac-368f118495fe\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prjbj"
Jul 13 19:30:27.315974 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:27.315807    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbl28\" (UniqueName: \"kubernetes.io/projected/40dbad3e-3816-4a6e-96ac-368f118495fe-kube-api-access-jbl28\") pod \"kuadrant-console-plugin-66b5774985-prjbj\" (UID: \"40dbad3e-3816-4a6e-96ac-368f118495fe\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prjbj"
Jul 13 19:30:27.416562 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:27.416526    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/40dbad3e-3816-4a6e-96ac-368f118495fe-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-prjbj\" (UID: \"40dbad3e-3816-4a6e-96ac-368f118495fe\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prjbj"
Jul 13 19:30:27.416770 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:27.416567    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/40dbad3e-3816-4a6e-96ac-368f118495fe-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-prjbj\" (UID: \"40dbad3e-3816-4a6e-96ac-368f118495fe\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prjbj"
Jul 13 19:30:27.416770 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:27.416605    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbl28\" (UniqueName: \"kubernetes.io/projected/40dbad3e-3816-4a6e-96ac-368f118495fe-kube-api-access-jbl28\") pod \"kuadrant-console-plugin-66b5774985-prjbj\" (UID: \"40dbad3e-3816-4a6e-96ac-368f118495fe\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prjbj"
Jul 13 19:30:27.417287 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:27.417260    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/40dbad3e-3816-4a6e-96ac-368f118495fe-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-prjbj\" (UID: \"40dbad3e-3816-4a6e-96ac-368f118495fe\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prjbj"
Jul 13 19:30:27.419047 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:27.419026    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/40dbad3e-3816-4a6e-96ac-368f118495fe-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-prjbj\" (UID: \"40dbad3e-3816-4a6e-96ac-368f118495fe\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prjbj"
Jul 13 19:30:27.424796 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:27.424772    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbl28\" (UniqueName: \"kubernetes.io/projected/40dbad3e-3816-4a6e-96ac-368f118495fe-kube-api-access-jbl28\") pod \"kuadrant-console-plugin-66b5774985-prjbj\" (UID: \"40dbad3e-3816-4a6e-96ac-368f118495fe\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prjbj"
Jul 13 19:30:27.560566 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:27.560474    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prjbj"
Jul 13 19:30:27.699527 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:27.699499    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-prjbj"]
Jul 13 19:30:27.702128 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:30:27.702090    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40dbad3e_3816_4a6e_96ac_368f118495fe.slice/crio-d3dcf87aef39934834d55c46d495035e3050d82656907dfd2e29a602cf7b492b WatchSource:0}: Error finding container d3dcf87aef39934834d55c46d495035e3050d82656907dfd2e29a602cf7b492b: Status 404 returned error can't find the container with id d3dcf87aef39934834d55c46d495035e3050d82656907dfd2e29a602cf7b492b
Jul 13 19:30:28.322835 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:28.322801    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prjbj" event={"ID":"40dbad3e-3816-4a6e-96ac-368f118495fe","Type":"ContainerStarted","Data":"d3dcf87aef39934834d55c46d495035e3050d82656907dfd2e29a602cf7b492b"}
Jul 13 19:30:33.301262 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:33.301045    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pxkvw"
Jul 13 19:30:51.444082 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:51.444044    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prjbj" event={"ID":"40dbad3e-3816-4a6e-96ac-368f118495fe","Type":"ContainerStarted","Data":"0831b376cabecaccda3100301791389afc1ca61b81769d3822f9f4e8734e2b6b"}
Jul 13 19:30:51.481879 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:30:51.481826    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prjbj" podStartSLOduration=1.040170078 podStartE2EDuration="24.481808712s" totalImagesPullingTime="23.441638634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:30:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:30:51.480759585 +0000 UTC m=+548.994636212" watchObservedRunningTime="2026-07-13 19:30:51.481808712 +0000 UTC m=+548.995685318"
Jul 13 19:31:24.795697 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:24.795661    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-cvmg5"]
Jul 13 19:31:24.826741 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:24.826707    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-cvmg5"]
Jul 13 19:31:24.826905 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:24.826836    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cvmg5"
Jul 13 19:31:24.829478 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:24.829458    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lq2xj\""
Jul 13 19:31:24.901601 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:24.901566    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mds7r\" (UniqueName: \"kubernetes.io/projected/c14c36b7-13a7-473a-bf8d-555ad97e4e4b-kube-api-access-mds7r\") pod \"authorino-58cd955675-cvmg5\" (UID: \"c14c36b7-13a7-473a-bf8d-555ad97e4e4b\") " pod="kuadrant-system/authorino-58cd955675-cvmg5"
Jul 13 19:31:25.002253 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:25.002214    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mds7r\" (UniqueName: \"kubernetes.io/projected/c14c36b7-13a7-473a-bf8d-555ad97e4e4b-kube-api-access-mds7r\") pod \"authorino-58cd955675-cvmg5\" (UID: \"c14c36b7-13a7-473a-bf8d-555ad97e4e4b\") " pod="kuadrant-system/authorino-58cd955675-cvmg5"
Jul 13 19:31:25.016226 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:25.016190    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-rk4h7"]
Jul 13 19:31:25.017871 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:25.017848    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mds7r\" (UniqueName: \"kubernetes.io/projected/c14c36b7-13a7-473a-bf8d-555ad97e4e4b-kube-api-access-mds7r\") pod \"authorino-58cd955675-cvmg5\" (UID: \"c14c36b7-13a7-473a-bf8d-555ad97e4e4b\") " pod="kuadrant-system/authorino-58cd955675-cvmg5"
Jul 13 19:31:25.018969 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:25.018953    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rk4h7"
Jul 13 19:31:25.030594 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:25.030563    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rk4h7"]
Jul 13 19:31:25.103121 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:25.103039    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp2pj\" (UniqueName: \"kubernetes.io/projected/ea072d28-7910-4d6c-a79a-d140fe3eb26d-kube-api-access-gp2pj\") pod \"authorino-75d556bd77-rk4h7\" (UID: \"ea072d28-7910-4d6c-a79a-d140fe3eb26d\") " pod="kuadrant-system/authorino-75d556bd77-rk4h7"
Jul 13 19:31:25.136276 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:25.136240    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cvmg5"
Jul 13 19:31:25.203746 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:25.203621    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp2pj\" (UniqueName: \"kubernetes.io/projected/ea072d28-7910-4d6c-a79a-d140fe3eb26d-kube-api-access-gp2pj\") pod \"authorino-75d556bd77-rk4h7\" (UID: \"ea072d28-7910-4d6c-a79a-d140fe3eb26d\") " pod="kuadrant-system/authorino-75d556bd77-rk4h7"
Jul 13 19:31:25.213693 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:25.213630    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp2pj\" (UniqueName: \"kubernetes.io/projected/ea072d28-7910-4d6c-a79a-d140fe3eb26d-kube-api-access-gp2pj\") pod \"authorino-75d556bd77-rk4h7\" (UID: \"ea072d28-7910-4d6c-a79a-d140fe3eb26d\") " pod="kuadrant-system/authorino-75d556bd77-rk4h7"
Jul 13 19:31:25.271463 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:25.271424    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-cvmg5"]
Jul 13 19:31:25.275612 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:31:25.275584    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc14c36b7_13a7_473a_bf8d_555ad97e4e4b.slice/crio-2b0f717dfff22e2812208f358a79d996043ef111e33ceb5a48e2c307b6d48b23 WatchSource:0}: Error finding container 2b0f717dfff22e2812208f358a79d996043ef111e33ceb5a48e2c307b6d48b23: Status 404 returned error can't find the container with id 2b0f717dfff22e2812208f358a79d996043ef111e33ceb5a48e2c307b6d48b23
Jul 13 19:31:25.334862 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:25.334827    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rk4h7"
Jul 13 19:31:25.471772 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:25.471747    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rk4h7"]
Jul 13 19:31:25.473606 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:31:25.473580    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea072d28_7910_4d6c_a79a_d140fe3eb26d.slice/crio-84c633deac1a702e24b523c9399f63503d87d4322b671bd022b1f605fc8c613d WatchSource:0}: Error finding container 84c633deac1a702e24b523c9399f63503d87d4322b671bd022b1f605fc8c613d: Status 404 returned error can't find the container with id 84c633deac1a702e24b523c9399f63503d87d4322b671bd022b1f605fc8c613d
Jul 13 19:31:25.607839 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:25.607802    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rk4h7" event={"ID":"ea072d28-7910-4d6c-a79a-d140fe3eb26d","Type":"ContainerStarted","Data":"84c633deac1a702e24b523c9399f63503d87d4322b671bd022b1f605fc8c613d"}
Jul 13 19:31:25.608893 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:25.608871    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cvmg5" event={"ID":"c14c36b7-13a7-473a-bf8d-555ad97e4e4b","Type":"ContainerStarted","Data":"2b0f717dfff22e2812208f358a79d996043ef111e33ceb5a48e2c307b6d48b23"}
Jul 13 19:31:29.633501 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:29.633466    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rk4h7" event={"ID":"ea072d28-7910-4d6c-a79a-d140fe3eb26d","Type":"ContainerStarted","Data":"1797d45873bd882f9cd44e002a9626119f7e923120067c1162fda93ba6452845"}
Jul 13 19:31:29.635044 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:29.635021    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cvmg5" event={"ID":"c14c36b7-13a7-473a-bf8d-555ad97e4e4b","Type":"ContainerStarted","Data":"0e6b259e7dbd57aa7d537dbdce1529bbe2d20e6c649394a7e2efec73f5accd79"}
Jul 13 19:31:29.650751 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:29.650692    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-rk4h7" podStartSLOduration=2.275353078 podStartE2EDuration="5.650677331s" totalImagesPullingTime="3.375324253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:31:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:31:29.649787933 +0000 UTC m=+587.163664540" watchObservedRunningTime="2026-07-13 19:31:29.650677331 +0000 UTC m=+587.164553936"
Jul 13 19:31:29.664931 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:29.664599    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-cvmg5" podStartSLOduration=2.102991007 podStartE2EDuration="5.66457494s" totalImagesPullingTime="3.561583933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:31:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:31:29.663227287 +0000 UTC m=+587.177103961" watchObservedRunningTime="2026-07-13 19:31:29.66457494 +0000 UTC m=+587.178451549"
Jul 13 19:31:29.687547 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:29.687510    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-cvmg5"]
Jul 13 19:31:31.643551 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:31.643492    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-cvmg5" podUID="c14c36b7-13a7-473a-bf8d-555ad97e4e4b" containerName="authorino" containerID="cri-o://0e6b259e7dbd57aa7d537dbdce1529bbe2d20e6c649394a7e2efec73f5accd79" gracePeriod=30
Jul 13 19:31:31.898393 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:31.898331    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cvmg5"
Jul 13 19:31:31.957444 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:31.957406    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mds7r\" (UniqueName: \"kubernetes.io/projected/c14c36b7-13a7-473a-bf8d-555ad97e4e4b-kube-api-access-mds7r\") pod \"c14c36b7-13a7-473a-bf8d-555ad97e4e4b\" (UID: \"c14c36b7-13a7-473a-bf8d-555ad97e4e4b\") "
Jul 13 19:31:31.959657 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:31.959612    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c14c36b7-13a7-473a-bf8d-555ad97e4e4b-kube-api-access-mds7r" (OuterVolumeSpecName: "kube-api-access-mds7r") pod "c14c36b7-13a7-473a-bf8d-555ad97e4e4b" (UID: "c14c36b7-13a7-473a-bf8d-555ad97e4e4b"). InnerVolumeSpecName "kube-api-access-mds7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:31:32.058433 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:32.058393    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mds7r\" (UniqueName: \"kubernetes.io/projected/c14c36b7-13a7-473a-bf8d-555ad97e4e4b-kube-api-access-mds7r\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:31:32.649520 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:32.649484    2573 generic.go:356] "Generic (PLEG): container finished" podID="c14c36b7-13a7-473a-bf8d-555ad97e4e4b" containerID="0e6b259e7dbd57aa7d537dbdce1529bbe2d20e6c649394a7e2efec73f5accd79" exitCode=0
Jul 13 19:31:32.649983 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:32.649527    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cvmg5"
Jul 13 19:31:32.649983 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:32.649553    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cvmg5" event={"ID":"c14c36b7-13a7-473a-bf8d-555ad97e4e4b","Type":"ContainerDied","Data":"0e6b259e7dbd57aa7d537dbdce1529bbe2d20e6c649394a7e2efec73f5accd79"}
Jul 13 19:31:32.649983 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:32.649591    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cvmg5" event={"ID":"c14c36b7-13a7-473a-bf8d-555ad97e4e4b","Type":"ContainerDied","Data":"2b0f717dfff22e2812208f358a79d996043ef111e33ceb5a48e2c307b6d48b23"}
Jul 13 19:31:32.649983 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:32.649613    2573 scope.go:117] "RemoveContainer" containerID="0e6b259e7dbd57aa7d537dbdce1529bbe2d20e6c649394a7e2efec73f5accd79"
Jul 13 19:31:32.658561 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:32.658542    2573 scope.go:117] "RemoveContainer" containerID="0e6b259e7dbd57aa7d537dbdce1529bbe2d20e6c649394a7e2efec73f5accd79"
Jul 13 19:31:32.658864 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:31:32.658846    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e6b259e7dbd57aa7d537dbdce1529bbe2d20e6c649394a7e2efec73f5accd79\": container with ID starting with 0e6b259e7dbd57aa7d537dbdce1529bbe2d20e6c649394a7e2efec73f5accd79 not found: ID does not exist" containerID="0e6b259e7dbd57aa7d537dbdce1529bbe2d20e6c649394a7e2efec73f5accd79"
Jul 13 19:31:32.658967 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:32.658872    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e6b259e7dbd57aa7d537dbdce1529bbe2d20e6c649394a7e2efec73f5accd79"} err="failed to get container status \"0e6b259e7dbd57aa7d537dbdce1529bbe2d20e6c649394a7e2efec73f5accd79\": rpc error: code = NotFound desc = could not find container \"0e6b259e7dbd57aa7d537dbdce1529bbe2d20e6c649394a7e2efec73f5accd79\": container with ID starting with 0e6b259e7dbd57aa7d537dbdce1529bbe2d20e6c649394a7e2efec73f5accd79 not found: ID does not exist"
Jul 13 19:31:32.670022 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:32.669994    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-cvmg5"]
Jul 13 19:31:32.674024 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:32.674001    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-cvmg5"]
Jul 13 19:31:33.048712 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:33.048678    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c14c36b7-13a7-473a-bf8d-555ad97e4e4b" path="/var/lib/kubelet/pods/c14c36b7-13a7-473a-bf8d-555ad97e4e4b/volumes"
Jul 13 19:31:42.891217 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:42.891192    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 19:31:42.891770 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:42.891750    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 19:31:42.893851 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:42.893829    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 19:31:42.894284 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:42.894264    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 19:31:57.808415 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:57.808374    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-m2z7d"]
Jul 13 19:31:57.809049 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:57.808916    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c14c36b7-13a7-473a-bf8d-555ad97e4e4b" containerName="authorino"
Jul 13 19:31:57.809049 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:57.808936    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c14c36b7-13a7-473a-bf8d-555ad97e4e4b" containerName="authorino"
Jul 13 19:31:57.809049 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:57.809036    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c14c36b7-13a7-473a-bf8d-555ad97e4e4b" containerName="authorino"
Jul 13 19:31:57.811600 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:57.811578    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-m2z7d"
Jul 13 19:31:57.819729 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:57.819703    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-m2z7d"]
Jul 13 19:31:57.884597 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:57.884561    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn4tr\" (UniqueName: \"kubernetes.io/projected/69f5eb5c-86f4-4ad5-9326-32aa08fca346-kube-api-access-sn4tr\") pod \"authorino-5d4b67c446-m2z7d\" (UID: \"69f5eb5c-86f4-4ad5-9326-32aa08fca346\") " pod="kuadrant-system/authorino-5d4b67c446-m2z7d"
Jul 13 19:31:57.985803 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:57.985759    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn4tr\" (UniqueName: \"kubernetes.io/projected/69f5eb5c-86f4-4ad5-9326-32aa08fca346-kube-api-access-sn4tr\") pod \"authorino-5d4b67c446-m2z7d\" (UID: \"69f5eb5c-86f4-4ad5-9326-32aa08fca346\") " pod="kuadrant-system/authorino-5d4b67c446-m2z7d"
Jul 13 19:31:57.995558 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:57.995534    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn4tr\" (UniqueName: \"kubernetes.io/projected/69f5eb5c-86f4-4ad5-9326-32aa08fca346-kube-api-access-sn4tr\") pod \"authorino-5d4b67c446-m2z7d\" (UID: \"69f5eb5c-86f4-4ad5-9326-32aa08fca346\") " pod="kuadrant-system/authorino-5d4b67c446-m2z7d"
Jul 13 19:31:58.038150 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:58.038114    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-m2z7d"]
Jul 13 19:31:58.038374 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:58.038363    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-m2z7d"
Jul 13 19:31:58.068015 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:58.067923    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6cd5dcc4d6-pkgt4"]
Jul 13 19:31:58.070298 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:58.070273    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cd5dcc4d6-pkgt4"
Jul 13 19:31:58.079529 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:58.079498    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6cd5dcc4d6-pkgt4"]
Jul 13 19:31:58.172582 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:58.172533    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-m2z7d"]
Jul 13 19:31:58.175654 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:31:58.175615    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69f5eb5c_86f4_4ad5_9326_32aa08fca346.slice/crio-ac2e5c67a1a3e1437b9bae21509fa9765741d7421c329fcd8771d4c57d3980b9 WatchSource:0}: Error finding container ac2e5c67a1a3e1437b9bae21509fa9765741d7421c329fcd8771d4c57d3980b9: Status 404 returned error can't find the container with id ac2e5c67a1a3e1437b9bae21509fa9765741d7421c329fcd8771d4c57d3980b9
Jul 13 19:31:58.186928 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:58.186899    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv5r2\" (UniqueName: \"kubernetes.io/projected/b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52-kube-api-access-jv5r2\") pod \"authorino-6cd5dcc4d6-pkgt4\" (UID: \"b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52\") " pod="kuadrant-system/authorino-6cd5dcc4d6-pkgt4"
Jul 13 19:31:58.288075 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:58.288044    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv5r2\" (UniqueName: \"kubernetes.io/projected/b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52-kube-api-access-jv5r2\") pod \"authorino-6cd5dcc4d6-pkgt4\" (UID: \"b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52\") " pod="kuadrant-system/authorino-6cd5dcc4d6-pkgt4"
Jul 13 19:31:58.297422 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:58.297395    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv5r2\" (UniqueName: \"kubernetes.io/projected/b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52-kube-api-access-jv5r2\") pod \"authorino-6cd5dcc4d6-pkgt4\" (UID: \"b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52\") " pod="kuadrant-system/authorino-6cd5dcc4d6-pkgt4"
Jul 13 19:31:58.382410 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:58.382326    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cd5dcc4d6-pkgt4"
Jul 13 19:31:58.514595 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:58.514571    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6cd5dcc4d6-pkgt4"]
Jul 13 19:31:58.516872 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:31:58.516847    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7a60ae0_f53e_4e12_ab65_1fbad9ad4d52.slice/crio-a06662abb2eafc55a8c626d54f894609a854bc14b08a161dab2aeb1be3afef33 WatchSource:0}: Error finding container a06662abb2eafc55a8c626d54f894609a854bc14b08a161dab2aeb1be3afef33: Status 404 returned error can't find the container with id a06662abb2eafc55a8c626d54f894609a854bc14b08a161dab2aeb1be3afef33
Jul 13 19:31:58.768767 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:58.768733    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-m2z7d" event={"ID":"69f5eb5c-86f4-4ad5-9326-32aa08fca346","Type":"ContainerStarted","Data":"ac2e5c67a1a3e1437b9bae21509fa9765741d7421c329fcd8771d4c57d3980b9"}
Jul 13 19:31:58.769971 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:58.769940    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cd5dcc4d6-pkgt4" event={"ID":"b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52","Type":"ContainerStarted","Data":"a06662abb2eafc55a8c626d54f894609a854bc14b08a161dab2aeb1be3afef33"}
Jul 13 19:31:59.776415 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:59.776372    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-m2z7d" event={"ID":"69f5eb5c-86f4-4ad5-9326-32aa08fca346","Type":"ContainerStarted","Data":"3279b24e46247b03b1e3e1851fe20c52f32c65b34ecd308c929aa0d801c996e0"}
Jul 13 19:31:59.776957 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:59.776430    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-m2z7d" podUID="69f5eb5c-86f4-4ad5-9326-32aa08fca346" containerName="authorino" containerID="cri-o://3279b24e46247b03b1e3e1851fe20c52f32c65b34ecd308c929aa0d801c996e0" gracePeriod=30
Jul 13 19:31:59.778030 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:59.778002    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cd5dcc4d6-pkgt4" event={"ID":"b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52","Type":"ContainerStarted","Data":"cc47c2830b31b2c6b27c89f33f22290daac69c1ce9e8de1d2a15a8a54100ec65"}
Jul 13 19:31:59.798448 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:59.798398    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-m2z7d" podStartSLOduration=2.097121106 podStartE2EDuration="2.798382542s" totalImagesPullingTime="701.261436ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:31:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:31:59.796328453 +0000 UTC m=+617.310205076" watchObservedRunningTime="2026-07-13 19:31:59.798382542 +0000 UTC m=+617.312259148"
Jul 13 19:31:59.822402 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:59.822351    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6cd5dcc4d6-pkgt4" podStartSLOduration=1.505167127 podStartE2EDuration="1.822332851s" totalImagesPullingTime="317.165724ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:31:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:31:59.821602997 +0000 UTC m=+617.335479676" watchObservedRunningTime="2026-07-13 19:31:59.822332851 +0000 UTC m=+617.336209457"
Jul 13 19:31:59.864236 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:59.864190    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rk4h7"]
Jul 13 19:31:59.864454 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:31:59.864426    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-rk4h7" podUID="ea072d28-7910-4d6c-a79a-d140fe3eb26d" containerName="authorino" containerID="cri-o://1797d45873bd882f9cd44e002a9626119f7e923120067c1162fda93ba6452845" gracePeriod=30
Jul 13 19:32:00.136862 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.136831    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-m2z7d"
Jul 13 19:32:00.141156 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.141129    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rk4h7"
Jul 13 19:32:00.202660 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.202596    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sn4tr\" (UniqueName: \"kubernetes.io/projected/69f5eb5c-86f4-4ad5-9326-32aa08fca346-kube-api-access-sn4tr\") pod \"69f5eb5c-86f4-4ad5-9326-32aa08fca346\" (UID: \"69f5eb5c-86f4-4ad5-9326-32aa08fca346\") "
Jul 13 19:32:00.204673 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.204647    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69f5eb5c-86f4-4ad5-9326-32aa08fca346-kube-api-access-sn4tr" (OuterVolumeSpecName: "kube-api-access-sn4tr") pod "69f5eb5c-86f4-4ad5-9326-32aa08fca346" (UID: "69f5eb5c-86f4-4ad5-9326-32aa08fca346"). InnerVolumeSpecName "kube-api-access-sn4tr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:32:00.277542 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.277506    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-ff67f5674-qh5sb"]
Jul 13 19:32:00.277882 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.277870    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69f5eb5c-86f4-4ad5-9326-32aa08fca346" containerName="authorino"
Jul 13 19:32:00.277923 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.277884    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="69f5eb5c-86f4-4ad5-9326-32aa08fca346" containerName="authorino"
Jul 13 19:32:00.277923 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.277904    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea072d28-7910-4d6c-a79a-d140fe3eb26d" containerName="authorino"
Jul 13 19:32:00.277923 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.277910    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea072d28-7910-4d6c-a79a-d140fe3eb26d" containerName="authorino"
Jul 13 19:32:00.278010 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.277965    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea072d28-7910-4d6c-a79a-d140fe3eb26d" containerName="authorino"
Jul 13 19:32:00.278010 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.277973    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="69f5eb5c-86f4-4ad5-9326-32aa08fca346" containerName="authorino"
Jul 13 19:32:00.280506 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.280489    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-ff67f5674-qh5sb"
Jul 13 19:32:00.283314 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.283293    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 13 19:32:00.292615 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.292578    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-ff67f5674-qh5sb"]
Jul 13 19:32:00.303831 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.303795    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gp2pj\" (UniqueName: \"kubernetes.io/projected/ea072d28-7910-4d6c-a79a-d140fe3eb26d-kube-api-access-gp2pj\") pod \"ea072d28-7910-4d6c-a79a-d140fe3eb26d\" (UID: \"ea072d28-7910-4d6c-a79a-d140fe3eb26d\") "
Jul 13 19:32:00.304056 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.304036    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sn4tr\" (UniqueName: \"kubernetes.io/projected/69f5eb5c-86f4-4ad5-9326-32aa08fca346-kube-api-access-sn4tr\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:32:00.306189 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.306161    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea072d28-7910-4d6c-a79a-d140fe3eb26d-kube-api-access-gp2pj" (OuterVolumeSpecName: "kube-api-access-gp2pj") pod "ea072d28-7910-4d6c-a79a-d140fe3eb26d" (UID: "ea072d28-7910-4d6c-a79a-d140fe3eb26d"). InnerVolumeSpecName "kube-api-access-gp2pj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:32:00.405070 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.404987    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncrkx\" (UniqueName: \"kubernetes.io/projected/b218e9c9-8256-4fad-b8ef-06cb139cfea9-kube-api-access-ncrkx\") pod \"authorino-ff67f5674-qh5sb\" (UID: \"b218e9c9-8256-4fad-b8ef-06cb139cfea9\") " pod="kuadrant-system/authorino-ff67f5674-qh5sb"
Jul 13 19:32:00.405070 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.405052    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b218e9c9-8256-4fad-b8ef-06cb139cfea9-tls-cert\") pod \"authorino-ff67f5674-qh5sb\" (UID: \"b218e9c9-8256-4fad-b8ef-06cb139cfea9\") " pod="kuadrant-system/authorino-ff67f5674-qh5sb"
Jul 13 19:32:00.405247 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.405104    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gp2pj\" (UniqueName: \"kubernetes.io/projected/ea072d28-7910-4d6c-a79a-d140fe3eb26d-kube-api-access-gp2pj\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:32:00.506270 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.506226    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b218e9c9-8256-4fad-b8ef-06cb139cfea9-tls-cert\") pod \"authorino-ff67f5674-qh5sb\" (UID: \"b218e9c9-8256-4fad-b8ef-06cb139cfea9\") " pod="kuadrant-system/authorino-ff67f5674-qh5sb"
Jul 13 19:32:00.506442 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.506317    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncrkx\" (UniqueName: \"kubernetes.io/projected/b218e9c9-8256-4fad-b8ef-06cb139cfea9-kube-api-access-ncrkx\") pod \"authorino-ff67f5674-qh5sb\" (UID: \"b218e9c9-8256-4fad-b8ef-06cb139cfea9\") " pod="kuadrant-system/authorino-ff67f5674-qh5sb"
Jul 13 19:32:00.508749 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.508729    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b218e9c9-8256-4fad-b8ef-06cb139cfea9-tls-cert\") pod \"authorino-ff67f5674-qh5sb\" (UID: \"b218e9c9-8256-4fad-b8ef-06cb139cfea9\") " pod="kuadrant-system/authorino-ff67f5674-qh5sb"
Jul 13 19:32:00.518328 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.518297    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncrkx\" (UniqueName: \"kubernetes.io/projected/b218e9c9-8256-4fad-b8ef-06cb139cfea9-kube-api-access-ncrkx\") pod \"authorino-ff67f5674-qh5sb\" (UID: \"b218e9c9-8256-4fad-b8ef-06cb139cfea9\") " pod="kuadrant-system/authorino-ff67f5674-qh5sb"
Jul 13 19:32:00.590159 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.590122    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-ff67f5674-qh5sb"
Jul 13 19:32:00.736286 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.736255    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-ff67f5674-qh5sb"]
Jul 13 19:32:00.738699 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:32:00.738670    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb218e9c9_8256_4fad_b8ef_06cb139cfea9.slice/crio-47bf51bc7ed9b7aa62a4c0783404cf77f79b188adfe8400be2f2abb43907fffb WatchSource:0}: Error finding container 47bf51bc7ed9b7aa62a4c0783404cf77f79b188adfe8400be2f2abb43907fffb: Status 404 returned error can't find the container with id 47bf51bc7ed9b7aa62a4c0783404cf77f79b188adfe8400be2f2abb43907fffb
Jul 13 19:32:00.784045 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.784008    2573 generic.go:356] "Generic (PLEG): container finished" podID="69f5eb5c-86f4-4ad5-9326-32aa08fca346" containerID="3279b24e46247b03b1e3e1851fe20c52f32c65b34ecd308c929aa0d801c996e0" exitCode=0
Jul 13 19:32:00.784497 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.784052    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-m2z7d"
Jul 13 19:32:00.784497 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.784094    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-m2z7d" event={"ID":"69f5eb5c-86f4-4ad5-9326-32aa08fca346","Type":"ContainerDied","Data":"3279b24e46247b03b1e3e1851fe20c52f32c65b34ecd308c929aa0d801c996e0"}
Jul 13 19:32:00.784497 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.784133    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-m2z7d" event={"ID":"69f5eb5c-86f4-4ad5-9326-32aa08fca346","Type":"ContainerDied","Data":"ac2e5c67a1a3e1437b9bae21509fa9765741d7421c329fcd8771d4c57d3980b9"}
Jul 13 19:32:00.784497 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.784153    2573 scope.go:117] "RemoveContainer" containerID="3279b24e46247b03b1e3e1851fe20c52f32c65b34ecd308c929aa0d801c996e0"
Jul 13 19:32:00.785531 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.785387    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-ff67f5674-qh5sb" event={"ID":"b218e9c9-8256-4fad-b8ef-06cb139cfea9","Type":"ContainerStarted","Data":"47bf51bc7ed9b7aa62a4c0783404cf77f79b188adfe8400be2f2abb43907fffb"}
Jul 13 19:32:00.786599 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.786579    2573 generic.go:356] "Generic (PLEG): container finished" podID="ea072d28-7910-4d6c-a79a-d140fe3eb26d" containerID="1797d45873bd882f9cd44e002a9626119f7e923120067c1162fda93ba6452845" exitCode=0
Jul 13 19:32:00.786707 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.786673    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rk4h7" event={"ID":"ea072d28-7910-4d6c-a79a-d140fe3eb26d","Type":"ContainerDied","Data":"1797d45873bd882f9cd44e002a9626119f7e923120067c1162fda93ba6452845"}
Jul 13 19:32:00.786764 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.786707    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rk4h7" event={"ID":"ea072d28-7910-4d6c-a79a-d140fe3eb26d","Type":"ContainerDied","Data":"84c633deac1a702e24b523c9399f63503d87d4322b671bd022b1f605fc8c613d"}
Jul 13 19:32:00.786846 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.786833    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rk4h7"
Jul 13 19:32:00.794116 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.793982    2573 scope.go:117] "RemoveContainer" containerID="3279b24e46247b03b1e3e1851fe20c52f32c65b34ecd308c929aa0d801c996e0"
Jul 13 19:32:00.794425 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:32:00.794402    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3279b24e46247b03b1e3e1851fe20c52f32c65b34ecd308c929aa0d801c996e0\": container with ID starting with 3279b24e46247b03b1e3e1851fe20c52f32c65b34ecd308c929aa0d801c996e0 not found: ID does not exist" containerID="3279b24e46247b03b1e3e1851fe20c52f32c65b34ecd308c929aa0d801c996e0"
Jul 13 19:32:00.794526 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.794435    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3279b24e46247b03b1e3e1851fe20c52f32c65b34ecd308c929aa0d801c996e0"} err="failed to get container status \"3279b24e46247b03b1e3e1851fe20c52f32c65b34ecd308c929aa0d801c996e0\": rpc error: code = NotFound desc = could not find container \"3279b24e46247b03b1e3e1851fe20c52f32c65b34ecd308c929aa0d801c996e0\": container with ID starting with 3279b24e46247b03b1e3e1851fe20c52f32c65b34ecd308c929aa0d801c996e0 not found: ID does not exist"
Jul 13 19:32:00.794526 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.794451    2573 scope.go:117] "RemoveContainer" containerID="1797d45873bd882f9cd44e002a9626119f7e923120067c1162fda93ba6452845"
Jul 13 19:32:00.802262 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.802243    2573 scope.go:117] "RemoveContainer" containerID="1797d45873bd882f9cd44e002a9626119f7e923120067c1162fda93ba6452845"
Jul 13 19:32:00.802536 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:32:00.802518    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1797d45873bd882f9cd44e002a9626119f7e923120067c1162fda93ba6452845\": container with ID starting with 1797d45873bd882f9cd44e002a9626119f7e923120067c1162fda93ba6452845 not found: ID does not exist" containerID="1797d45873bd882f9cd44e002a9626119f7e923120067c1162fda93ba6452845"
Jul 13 19:32:00.802583 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.802543    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1797d45873bd882f9cd44e002a9626119f7e923120067c1162fda93ba6452845"} err="failed to get container status \"1797d45873bd882f9cd44e002a9626119f7e923120067c1162fda93ba6452845\": rpc error: code = NotFound desc = could not find container \"1797d45873bd882f9cd44e002a9626119f7e923120067c1162fda93ba6452845\": container with ID starting with 1797d45873bd882f9cd44e002a9626119f7e923120067c1162fda93ba6452845 not found: ID does not exist"
Jul 13 19:32:00.811818 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.811788    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rk4h7"]
Jul 13 19:32:00.816470 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.816445    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rk4h7"]
Jul 13 19:32:00.830764 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.830735    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-m2z7d"]
Jul 13 19:32:00.840057 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:00.840034    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-m2z7d"]
Jul 13 19:32:01.048517 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:01.048483    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69f5eb5c-86f4-4ad5-9326-32aa08fca346" path="/var/lib/kubelet/pods/69f5eb5c-86f4-4ad5-9326-32aa08fca346/volumes"
Jul 13 19:32:01.048860 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:01.048846    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea072d28-7910-4d6c-a79a-d140fe3eb26d" path="/var/lib/kubelet/pods/ea072d28-7910-4d6c-a79a-d140fe3eb26d/volumes"
Jul 13 19:32:01.792566 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:01.792522    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-ff67f5674-qh5sb" event={"ID":"b218e9c9-8256-4fad-b8ef-06cb139cfea9","Type":"ContainerStarted","Data":"64502ad2c2079d792ec5b84164dfe9eac72bfe6fbf6e6c144e7009b6ec771e7c"}
Jul 13 19:32:01.809936 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:01.809881    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-ff67f5674-qh5sb" podStartSLOduration=1.431030273 podStartE2EDuration="1.809863474s" totalImagesPullingTime="378.833201ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:32:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:32:01.808592244 +0000 UTC m=+619.322468850" watchObservedRunningTime="2026-07-13 19:32:01.809863474 +0000 UTC m=+619.323740079"
Jul 13 19:32:01.840589 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:01.840549    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6cd5dcc4d6-pkgt4"]
Jul 13 19:32:01.840773 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:01.840731    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6cd5dcc4d6-pkgt4" podUID="b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52" containerName="authorino" containerID="cri-o://cc47c2830b31b2c6b27c89f33f22290daac69c1ce9e8de1d2a15a8a54100ec65" gracePeriod=30
Jul 13 19:32:02.097081 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:02.097039    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cd5dcc4d6-pkgt4"
Jul 13 19:32:02.222013 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:02.221977    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jv5r2\" (UniqueName: \"kubernetes.io/projected/b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52-kube-api-access-jv5r2\") pod \"b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52\" (UID: \"b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52\") "
Jul 13 19:32:02.224067 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:02.224038    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52-kube-api-access-jv5r2" (OuterVolumeSpecName: "kube-api-access-jv5r2") pod "b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52" (UID: "b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52"). InnerVolumeSpecName "kube-api-access-jv5r2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:32:02.323455 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:02.323415    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jv5r2\" (UniqueName: \"kubernetes.io/projected/b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52-kube-api-access-jv5r2\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:32:02.801038 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:02.800998    2573 generic.go:356] "Generic (PLEG): container finished" podID="b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52" containerID="cc47c2830b31b2c6b27c89f33f22290daac69c1ce9e8de1d2a15a8a54100ec65" exitCode=0
Jul 13 19:32:02.801504 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:02.801043    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cd5dcc4d6-pkgt4"
Jul 13 19:32:02.801504 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:02.801080    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cd5dcc4d6-pkgt4" event={"ID":"b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52","Type":"ContainerDied","Data":"cc47c2830b31b2c6b27c89f33f22290daac69c1ce9e8de1d2a15a8a54100ec65"}
Jul 13 19:32:02.801504 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:02.801120    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cd5dcc4d6-pkgt4" event={"ID":"b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52","Type":"ContainerDied","Data":"a06662abb2eafc55a8c626d54f894609a854bc14b08a161dab2aeb1be3afef33"}
Jul 13 19:32:02.801504 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:02.801136    2573 scope.go:117] "RemoveContainer" containerID="cc47c2830b31b2c6b27c89f33f22290daac69c1ce9e8de1d2a15a8a54100ec65"
Jul 13 19:32:02.809879 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:02.809861    2573 scope.go:117] "RemoveContainer" containerID="cc47c2830b31b2c6b27c89f33f22290daac69c1ce9e8de1d2a15a8a54100ec65"
Jul 13 19:32:02.810133 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:32:02.810114    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc47c2830b31b2c6b27c89f33f22290daac69c1ce9e8de1d2a15a8a54100ec65\": container with ID starting with cc47c2830b31b2c6b27c89f33f22290daac69c1ce9e8de1d2a15a8a54100ec65 not found: ID does not exist" containerID="cc47c2830b31b2c6b27c89f33f22290daac69c1ce9e8de1d2a15a8a54100ec65"
Jul 13 19:32:02.810186 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:02.810140    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc47c2830b31b2c6b27c89f33f22290daac69c1ce9e8de1d2a15a8a54100ec65"} err="failed to get container status \"cc47c2830b31b2c6b27c89f33f22290daac69c1ce9e8de1d2a15a8a54100ec65\": rpc error: code = NotFound desc = could not find container \"cc47c2830b31b2c6b27c89f33f22290daac69c1ce9e8de1d2a15a8a54100ec65\": container with ID starting with cc47c2830b31b2c6b27c89f33f22290daac69c1ce9e8de1d2a15a8a54100ec65 not found: ID does not exist"
Jul 13 19:32:02.822931 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:02.822900    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6cd5dcc4d6-pkgt4"]
Jul 13 19:32:02.829111 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:02.829085    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6cd5dcc4d6-pkgt4"]
Jul 13 19:32:03.050856 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:32:03.050763    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52" path="/var/lib/kubelet/pods/b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52/volumes"
Jul 13 19:33:28.304011 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.303976    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"]
Jul 13 19:33:28.304408 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.304319    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52" containerName="authorino"
Jul 13 19:33:28.304408 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.304331    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52" containerName="authorino"
Jul 13 19:33:28.304408 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.304394    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7a60ae0-f53e-4e12-ab65-1fbad9ad4d52" containerName="authorino"
Jul 13 19:33:28.307352 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.307333    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:28.310536 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.310513    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 13 19:33:28.311787 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.311733    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 13 19:33:28.311787 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.311752    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4wg86\""
Jul 13 19:33:28.312003 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.311848    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 13 19:33:28.319699 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.319674    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"]
Jul 13 19:33:28.352946 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.352905    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxnzx\" (UniqueName: \"kubernetes.io/projected/6f92caa9-f1be-4633-b300-ca3f1f874d45-kube-api-access-qxnzx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9\" (UID: \"6f92caa9-f1be-4633-b300-ca3f1f874d45\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:28.353113 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.352954    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f92caa9-f1be-4633-b300-ca3f1f874d45-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9\" (UID: \"6f92caa9-f1be-4633-b300-ca3f1f874d45\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:28.353113 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.352984    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f92caa9-f1be-4633-b300-ca3f1f874d45-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9\" (UID: \"6f92caa9-f1be-4633-b300-ca3f1f874d45\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:28.353113 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.353068    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f92caa9-f1be-4633-b300-ca3f1f874d45-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9\" (UID: \"6f92caa9-f1be-4633-b300-ca3f1f874d45\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:28.353241 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.353110    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f92caa9-f1be-4633-b300-ca3f1f874d45-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9\" (UID: \"6f92caa9-f1be-4633-b300-ca3f1f874d45\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:28.353241 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.353134    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f92caa9-f1be-4633-b300-ca3f1f874d45-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9\" (UID: \"6f92caa9-f1be-4633-b300-ca3f1f874d45\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:28.454049 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.454011    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxnzx\" (UniqueName: \"kubernetes.io/projected/6f92caa9-f1be-4633-b300-ca3f1f874d45-kube-api-access-qxnzx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9\" (UID: \"6f92caa9-f1be-4633-b300-ca3f1f874d45\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:28.454229 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.454062    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f92caa9-f1be-4633-b300-ca3f1f874d45-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9\" (UID: \"6f92caa9-f1be-4633-b300-ca3f1f874d45\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:28.454269 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.454238    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f92caa9-f1be-4633-b300-ca3f1f874d45-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9\" (UID: \"6f92caa9-f1be-4633-b300-ca3f1f874d45\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:28.454329 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.454312    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f92caa9-f1be-4633-b300-ca3f1f874d45-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9\" (UID: \"6f92caa9-f1be-4633-b300-ca3f1f874d45\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:28.454376 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.454352    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f92caa9-f1be-4633-b300-ca3f1f874d45-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9\" (UID: \"6f92caa9-f1be-4633-b300-ca3f1f874d45\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:28.454427 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.454379    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f92caa9-f1be-4633-b300-ca3f1f874d45-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9\" (UID: \"6f92caa9-f1be-4633-b300-ca3f1f874d45\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:28.454427 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.454415    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f92caa9-f1be-4633-b300-ca3f1f874d45-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9\" (UID: \"6f92caa9-f1be-4633-b300-ca3f1f874d45\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:28.454808 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.454779    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f92caa9-f1be-4633-b300-ca3f1f874d45-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9\" (UID: \"6f92caa9-f1be-4633-b300-ca3f1f874d45\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:28.454886 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.454805    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f92caa9-f1be-4633-b300-ca3f1f874d45-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9\" (UID: \"6f92caa9-f1be-4633-b300-ca3f1f874d45\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:28.456666 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.456626    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f92caa9-f1be-4633-b300-ca3f1f874d45-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9\" (UID: \"6f92caa9-f1be-4633-b300-ca3f1f874d45\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:28.457016 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.456996    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f92caa9-f1be-4633-b300-ca3f1f874d45-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9\" (UID: \"6f92caa9-f1be-4633-b300-ca3f1f874d45\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:28.462325 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.462301    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxnzx\" (UniqueName: \"kubernetes.io/projected/6f92caa9-f1be-4633-b300-ca3f1f874d45-kube-api-access-qxnzx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9\" (UID: \"6f92caa9-f1be-4633-b300-ca3f1f874d45\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:28.618208 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.618110    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:28.774109 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.772286    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"]
Jul 13 19:33:28.776797 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:28.776768    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 19:33:29.189031 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:29.188989    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9" event={"ID":"6f92caa9-f1be-4633-b300-ca3f1f874d45","Type":"ContainerStarted","Data":"964aa9e824757c162ca7358adab457a63f46276887fb9e1cd23ed0ddd573ade7"}
Jul 13 19:33:30.197736 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:30.197699    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9" event={"ID":"6f92caa9-f1be-4633-b300-ca3f1f874d45","Type":"ContainerStarted","Data":"4bda9fc80264c21360df5bec56d4daabff8532a03ac03b5f8589512ec86ceaaf"}
Jul 13 19:33:30.219701 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:30.219620    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9" podStartSLOduration=1.192483202 podStartE2EDuration="2.219602036s" totalImagesPullingTime="1.027118834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:33:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:33:30.218397976 +0000 UTC m=+707.732274585" watchObservedRunningTime="2026-07-13 19:33:30.219602036 +0000 UTC m=+707.733478642"
Jul 13 19:33:38.619280 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:38.619180    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:38.619280 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:38.619263    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:38.631758 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:38.631732    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:39.249439 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:39.249413    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9"
Jul 13 19:33:59.982965 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:59.982928    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"]
Jul 13 19:33:59.988427 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:59.988405    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:33:59.991135 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:59.991100    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 13 19:33:59.998332 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:33:59.998307    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"]
Jul 13 19:34:00.045432 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:00.045393    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8d34aef-a8fd-4b33-a587-f619f1e62d59-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7\" (UID: \"a8d34aef-a8fd-4b33-a587-f619f1e62d59\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:00.045612 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:00.045446    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8d34aef-a8fd-4b33-a587-f619f1e62d59-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7\" (UID: \"a8d34aef-a8fd-4b33-a587-f619f1e62d59\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:00.045612 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:00.045482    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8d34aef-a8fd-4b33-a587-f619f1e62d59-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7\" (UID: \"a8d34aef-a8fd-4b33-a587-f619f1e62d59\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:00.045612 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:00.045503    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mbm4\" (UniqueName: \"kubernetes.io/projected/a8d34aef-a8fd-4b33-a587-f619f1e62d59-kube-api-access-8mbm4\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7\" (UID: \"a8d34aef-a8fd-4b33-a587-f619f1e62d59\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:00.045612 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:00.045524    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8d34aef-a8fd-4b33-a587-f619f1e62d59-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7\" (UID: \"a8d34aef-a8fd-4b33-a587-f619f1e62d59\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:00.045612 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:00.045591    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8d34aef-a8fd-4b33-a587-f619f1e62d59-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7\" (UID: \"a8d34aef-a8fd-4b33-a587-f619f1e62d59\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:00.146868 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:00.146814    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8d34aef-a8fd-4b33-a587-f619f1e62d59-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7\" (UID: \"a8d34aef-a8fd-4b33-a587-f619f1e62d59\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:00.147052 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:00.146911    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8d34aef-a8fd-4b33-a587-f619f1e62d59-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7\" (UID: \"a8d34aef-a8fd-4b33-a587-f619f1e62d59\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:00.147052 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:00.146952    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8d34aef-a8fd-4b33-a587-f619f1e62d59-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7\" (UID: \"a8d34aef-a8fd-4b33-a587-f619f1e62d59\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:00.147052 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:00.146988    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mbm4\" (UniqueName: \"kubernetes.io/projected/a8d34aef-a8fd-4b33-a587-f619f1e62d59-kube-api-access-8mbm4\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7\" (UID: \"a8d34aef-a8fd-4b33-a587-f619f1e62d59\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:00.147052 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:00.147021    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8d34aef-a8fd-4b33-a587-f619f1e62d59-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7\" (UID: \"a8d34aef-a8fd-4b33-a587-f619f1e62d59\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:00.147288 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:00.147072    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8d34aef-a8fd-4b33-a587-f619f1e62d59-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7\" (UID: \"a8d34aef-a8fd-4b33-a587-f619f1e62d59\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:00.147370 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:00.147345    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8d34aef-a8fd-4b33-a587-f619f1e62d59-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7\" (UID: \"a8d34aef-a8fd-4b33-a587-f619f1e62d59\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:00.147429 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:00.147367    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8d34aef-a8fd-4b33-a587-f619f1e62d59-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7\" (UID: \"a8d34aef-a8fd-4b33-a587-f619f1e62d59\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:00.147482 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:00.147442    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8d34aef-a8fd-4b33-a587-f619f1e62d59-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7\" (UID: \"a8d34aef-a8fd-4b33-a587-f619f1e62d59\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:00.149174 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:00.149149    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8d34aef-a8fd-4b33-a587-f619f1e62d59-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7\" (UID: \"a8d34aef-a8fd-4b33-a587-f619f1e62d59\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:00.149476 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:00.149459    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8d34aef-a8fd-4b33-a587-f619f1e62d59-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7\" (UID: \"a8d34aef-a8fd-4b33-a587-f619f1e62d59\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:00.154886 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:00.154861    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mbm4\" (UniqueName: \"kubernetes.io/projected/a8d34aef-a8fd-4b33-a587-f619f1e62d59-kube-api-access-8mbm4\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7\" (UID: \"a8d34aef-a8fd-4b33-a587-f619f1e62d59\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:00.297967 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:00.297872    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:00.434596 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:00.434570    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"]
Jul 13 19:34:00.436762 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:34:00.436734    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8d34aef_a8fd_4b33_a587_f619f1e62d59.slice/crio-50d63a76a782336c27c0c1451bda136759ec2f3a9d946751e57d8ac631cb757f WatchSource:0}: Error finding container 50d63a76a782336c27c0c1451bda136759ec2f3a9d946751e57d8ac631cb757f: Status 404 returned error can't find the container with id 50d63a76a782336c27c0c1451bda136759ec2f3a9d946751e57d8ac631cb757f
Jul 13 19:34:01.340260 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:01.340227    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7" event={"ID":"a8d34aef-a8fd-4b33-a587-f619f1e62d59","Type":"ContainerStarted","Data":"ea968944ed23984fc5328ab25ffebf76b4e3da7ba609706d4437f3bcaa006627"}
Jul 13 19:34:01.340654 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:01.340272    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7" event={"ID":"a8d34aef-a8fd-4b33-a587-f619f1e62d59","Type":"ContainerStarted","Data":"50d63a76a782336c27c0c1451bda136759ec2f3a9d946751e57d8ac631cb757f"}
Jul 13 19:34:01.360008 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:01.359953    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7" podStartSLOduration=2.156205483 podStartE2EDuration="2.359935736s" totalImagesPullingTime="203.730253ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:33:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:34:01.35738811 +0000 UTC m=+738.871264729" watchObservedRunningTime="2026-07-13 19:34:01.359935736 +0000 UTC m=+738.873812342"
Jul 13 19:34:10.298996 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:10.298957    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:10.299487 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:10.299013    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:10.311344 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:10.311316    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:10.393514 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:10.393484    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7"
Jul 13 19:34:56.387884 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:56.387843    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6dc479448c-rsbcn"]
Jul 13 19:34:56.391251 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:56.391230    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dc479448c-rsbcn"
Jul 13 19:34:56.400030 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:56.399990    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dc479448c-rsbcn"]
Jul 13 19:34:56.445181 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:56.445135    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-852v8\" (UniqueName: \"kubernetes.io/projected/d751e3ec-82ca-4020-8c9a-922ba248b1ca-kube-api-access-852v8\") pod \"authorino-6dc479448c-rsbcn\" (UID: \"d751e3ec-82ca-4020-8c9a-922ba248b1ca\") " pod="kuadrant-system/authorino-6dc479448c-rsbcn"
Jul 13 19:34:56.445350 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:56.445216    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d751e3ec-82ca-4020-8c9a-922ba248b1ca-tls-cert\") pod \"authorino-6dc479448c-rsbcn\" (UID: \"d751e3ec-82ca-4020-8c9a-922ba248b1ca\") " pod="kuadrant-system/authorino-6dc479448c-rsbcn"
Jul 13 19:34:56.546370 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:56.546335    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-852v8\" (UniqueName: \"kubernetes.io/projected/d751e3ec-82ca-4020-8c9a-922ba248b1ca-kube-api-access-852v8\") pod \"authorino-6dc479448c-rsbcn\" (UID: \"d751e3ec-82ca-4020-8c9a-922ba248b1ca\") " pod="kuadrant-system/authorino-6dc479448c-rsbcn"
Jul 13 19:34:56.546527 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:56.546399    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d751e3ec-82ca-4020-8c9a-922ba248b1ca-tls-cert\") pod \"authorino-6dc479448c-rsbcn\" (UID: \"d751e3ec-82ca-4020-8c9a-922ba248b1ca\") " pod="kuadrant-system/authorino-6dc479448c-rsbcn"
Jul 13 19:34:56.548830 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:56.548812    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d751e3ec-82ca-4020-8c9a-922ba248b1ca-tls-cert\") pod \"authorino-6dc479448c-rsbcn\" (UID: \"d751e3ec-82ca-4020-8c9a-922ba248b1ca\") " pod="kuadrant-system/authorino-6dc479448c-rsbcn"
Jul 13 19:34:56.554969 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:56.554942    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-852v8\" (UniqueName: \"kubernetes.io/projected/d751e3ec-82ca-4020-8c9a-922ba248b1ca-kube-api-access-852v8\") pod \"authorino-6dc479448c-rsbcn\" (UID: \"d751e3ec-82ca-4020-8c9a-922ba248b1ca\") " pod="kuadrant-system/authorino-6dc479448c-rsbcn"
Jul 13 19:34:56.700821 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:56.700789    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dc479448c-rsbcn"
Jul 13 19:34:56.831487 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:56.831460    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dc479448c-rsbcn"]
Jul 13 19:34:56.833769 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:34:56.833741    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd751e3ec_82ca_4020_8c9a_922ba248b1ca.slice/crio-1666d9e7ca57db5cf07dc4224e105b17a5f8b42b0ce52e6c413ecd8f7e54f560 WatchSource:0}: Error finding container 1666d9e7ca57db5cf07dc4224e105b17a5f8b42b0ce52e6c413ecd8f7e54f560: Status 404 returned error can't find the container with id 1666d9e7ca57db5cf07dc4224e105b17a5f8b42b0ce52e6c413ecd8f7e54f560
Jul 13 19:34:57.600084 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:57.600041    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dc479448c-rsbcn" event={"ID":"d751e3ec-82ca-4020-8c9a-922ba248b1ca","Type":"ContainerStarted","Data":"9daa8b5c8bbd1e02e356798a9000c6befebf3392b23d7c0e2c409a5c2f4eadea"}
Jul 13 19:34:57.600084 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:57.600086    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dc479448c-rsbcn" event={"ID":"d751e3ec-82ca-4020-8c9a-922ba248b1ca","Type":"ContainerStarted","Data":"1666d9e7ca57db5cf07dc4224e105b17a5f8b42b0ce52e6c413ecd8f7e54f560"}
Jul 13 19:34:57.649908 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:57.649781    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6dc479448c-rsbcn" podStartSLOduration=1.095389908 podStartE2EDuration="1.649756235s" totalImagesPullingTime="554.366327ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:34:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:34:57.61657332 +0000 UTC m=+795.130449925" watchObservedRunningTime="2026-07-13 19:34:57.649756235 +0000 UTC m=+795.163632842"
Jul 13 19:34:57.650393 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:57.650367    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-ff67f5674-qh5sb"]
Jul 13 19:34:57.650614 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:57.650589    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-ff67f5674-qh5sb" podUID="b218e9c9-8256-4fad-b8ef-06cb139cfea9" containerName="authorino" containerID="cri-o://64502ad2c2079d792ec5b84164dfe9eac72bfe6fbf6e6c144e7009b6ec771e7c" gracePeriod=30
Jul 13 19:34:57.914464 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:57.914396    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-ff67f5674-qh5sb"
Jul 13 19:34:57.962158 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:57.962120    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b218e9c9-8256-4fad-b8ef-06cb139cfea9-tls-cert\") pod \"b218e9c9-8256-4fad-b8ef-06cb139cfea9\" (UID: \"b218e9c9-8256-4fad-b8ef-06cb139cfea9\") "
Jul 13 19:34:57.962337 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:57.962241    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncrkx\" (UniqueName: \"kubernetes.io/projected/b218e9c9-8256-4fad-b8ef-06cb139cfea9-kube-api-access-ncrkx\") pod \"b218e9c9-8256-4fad-b8ef-06cb139cfea9\" (UID: \"b218e9c9-8256-4fad-b8ef-06cb139cfea9\") "
Jul 13 19:34:57.964246 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:57.964209    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b218e9c9-8256-4fad-b8ef-06cb139cfea9-kube-api-access-ncrkx" (OuterVolumeSpecName: "kube-api-access-ncrkx") pod "b218e9c9-8256-4fad-b8ef-06cb139cfea9" (UID: "b218e9c9-8256-4fad-b8ef-06cb139cfea9"). InnerVolumeSpecName "kube-api-access-ncrkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:34:57.981604 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:57.975837    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b218e9c9-8256-4fad-b8ef-06cb139cfea9-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "b218e9c9-8256-4fad-b8ef-06cb139cfea9" (UID: "b218e9c9-8256-4fad-b8ef-06cb139cfea9"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:34:58.063576 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:58.063532    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncrkx\" (UniqueName: \"kubernetes.io/projected/b218e9c9-8256-4fad-b8ef-06cb139cfea9-kube-api-access-ncrkx\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:34:58.063576 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:58.063564    2573 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b218e9c9-8256-4fad-b8ef-06cb139cfea9-tls-cert\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:34:58.607358 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:58.607321    2573 generic.go:356] "Generic (PLEG): container finished" podID="b218e9c9-8256-4fad-b8ef-06cb139cfea9" containerID="64502ad2c2079d792ec5b84164dfe9eac72bfe6fbf6e6c144e7009b6ec771e7c" exitCode=0
Jul 13 19:34:58.607842 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:58.607371    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-ff67f5674-qh5sb"
Jul 13 19:34:58.607842 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:58.607361    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-ff67f5674-qh5sb" event={"ID":"b218e9c9-8256-4fad-b8ef-06cb139cfea9","Type":"ContainerDied","Data":"64502ad2c2079d792ec5b84164dfe9eac72bfe6fbf6e6c144e7009b6ec771e7c"}
Jul 13 19:34:58.607842 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:58.607493    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-ff67f5674-qh5sb" event={"ID":"b218e9c9-8256-4fad-b8ef-06cb139cfea9","Type":"ContainerDied","Data":"47bf51bc7ed9b7aa62a4c0783404cf77f79b188adfe8400be2f2abb43907fffb"}
Jul 13 19:34:58.607842 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:58.607539    2573 scope.go:117] "RemoveContainer" containerID="64502ad2c2079d792ec5b84164dfe9eac72bfe6fbf6e6c144e7009b6ec771e7c"
Jul 13 19:34:58.617128 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:58.617074    2573 scope.go:117] "RemoveContainer" containerID="64502ad2c2079d792ec5b84164dfe9eac72bfe6fbf6e6c144e7009b6ec771e7c"
Jul 13 19:34:58.617474 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:34:58.617451    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64502ad2c2079d792ec5b84164dfe9eac72bfe6fbf6e6c144e7009b6ec771e7c\": container with ID starting with 64502ad2c2079d792ec5b84164dfe9eac72bfe6fbf6e6c144e7009b6ec771e7c not found: ID does not exist" containerID="64502ad2c2079d792ec5b84164dfe9eac72bfe6fbf6e6c144e7009b6ec771e7c"
Jul 13 19:34:58.617583 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:58.617480    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64502ad2c2079d792ec5b84164dfe9eac72bfe6fbf6e6c144e7009b6ec771e7c"} err="failed to get container status \"64502ad2c2079d792ec5b84164dfe9eac72bfe6fbf6e6c144e7009b6ec771e7c\": rpc error: code = NotFound desc = could not find container \"64502ad2c2079d792ec5b84164dfe9eac72bfe6fbf6e6c144e7009b6ec771e7c\": container with ID starting with 64502ad2c2079d792ec5b84164dfe9eac72bfe6fbf6e6c144e7009b6ec771e7c not found: ID does not exist"
Jul 13 19:34:58.630037 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:58.630001    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-ff67f5674-qh5sb"]
Jul 13 19:34:58.633474 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:58.633445    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-ff67f5674-qh5sb"]
Jul 13 19:34:59.049685 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:34:59.049648    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b218e9c9-8256-4fad-b8ef-06cb139cfea9" path="/var/lib/kubelet/pods/b218e9c9-8256-4fad-b8ef-06cb139cfea9/volumes"
Jul 13 19:36:42.919061 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:36:42.918979    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 19:36:42.920272 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:36:42.920254    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 19:36:42.921601 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:36:42.921583    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 19:36:42.922806 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:36:42.922789    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 19:41:42.946961 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:41:42.946927    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 19:41:42.949291 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:41:42.949271    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 19:41:42.949531 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:41:42.949515    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 19:41:42.951793 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:41:42.951771    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 19:45:00.145083 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:45:00.145049    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rx4qh"]
Jul 13 19:45:00.147716 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:45:00.145383    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b218e9c9-8256-4fad-b8ef-06cb139cfea9" containerName="authorino"
Jul 13 19:45:00.147716 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:45:00.145393    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b218e9c9-8256-4fad-b8ef-06cb139cfea9" containerName="authorino"
Jul 13 19:45:00.147716 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:45:00.145448    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b218e9c9-8256-4fad-b8ef-06cb139cfea9" containerName="authorino"
Jul 13 19:45:00.148713 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:45:00.148696    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rx4qh"
Jul 13 19:45:00.151699 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:45:00.151678    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q5n9g\""
Jul 13 19:45:00.158081 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:45:00.158064    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rx4qh"
Jul 13 19:45:00.165958 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:45:00.165928    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rx4qh"]
Jul 13 19:45:00.293665 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:45:00.293614    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rx4qh"]
Jul 13 19:45:00.296210 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:45:00.296175    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5da24b7_3b69_483d_90af_01daac639565.slice/crio-dde73ee2e638ed5d124dc2036e5879093ffa1616f27ac9c50e44bba942a3f52b WatchSource:0}: Error finding container dde73ee2e638ed5d124dc2036e5879093ffa1616f27ac9c50e44bba942a3f52b: Status 404 returned error can't find the container with id dde73ee2e638ed5d124dc2036e5879093ffa1616f27ac9c50e44bba942a3f52b
Jul 13 19:45:00.298135 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:45:00.298119    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 19:45:00.394571 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:45:00.394524    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rx4qh" event={"ID":"c5da24b7-3b69-483d-90af-01daac639565","Type":"ContainerStarted","Data":"dde73ee2e638ed5d124dc2036e5879093ffa1616f27ac9c50e44bba942a3f52b"}
Jul 13 19:45:03.413403 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:45:03.413307    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rx4qh" event={"ID":"c5da24b7-3b69-483d-90af-01daac639565","Type":"ContainerStarted","Data":"d12d4ca7e9193bb024d596d03c0c114a1ea5f4d28611f75fb8f942d3b0d194f8"}
Jul 13 19:45:03.428687 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:45:03.428621    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rx4qh" podStartSLOduration=1.055718063 podStartE2EDuration="3.428602683s" totalImagesPullingTime="2.37288462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:45:03.427460069 +0000 UTC m=+1400.941336675" watchObservedRunningTime="2026-07-13 19:45:03.428602683 +0000 UTC m=+1400.942479290"
Jul 13 19:45:04.419466 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:45:04.419431    2573 generic.go:356] "Generic (PLEG): container finished" podID="c5da24b7-3b69-483d-90af-01daac639565" containerID="d12d4ca7e9193bb024d596d03c0c114a1ea5f4d28611f75fb8f942d3b0d194f8" exitCode=0
Jul 13 19:45:04.419882 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:45:04.419521    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rx4qh" event={"ID":"c5da24b7-3b69-483d-90af-01daac639565","Type":"ContainerDied","Data":"d12d4ca7e9193bb024d596d03c0c114a1ea5f4d28611f75fb8f942d3b0d194f8"}
Jul 13 19:45:05.555322 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:45:05.555288    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rx4qh"
Jul 13 19:45:06.431014 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:45:06.430984    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rx4qh"
Jul 13 19:45:06.431186 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:45:06.430976    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732865-rx4qh" event={"ID":"c5da24b7-3b69-483d-90af-01daac639565","Type":"ContainerDied","Data":"dde73ee2e638ed5d124dc2036e5879093ffa1616f27ac9c50e44bba942a3f52b"}
Jul 13 19:45:06.431186 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:45:06.431125    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dde73ee2e638ed5d124dc2036e5879093ffa1616f27ac9c50e44bba942a3f52b"
Jul 13 19:46:42.978224 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:46:42.978188    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 19:46:42.979703 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:46:42.979681    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 19:46:42.980595 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:46:42.980574    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 19:46:42.982050 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:46:42.982024    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 19:48:46.976550 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:48:46.976513    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94"]
Jul 13 19:48:46.977028 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:48:46.976876    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5da24b7-3b69-483d-90af-01daac639565" containerName="cleanup"
Jul 13 19:48:46.977028 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:48:46.976887    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5da24b7-3b69-483d-90af-01daac639565" containerName="cleanup"
Jul 13 19:48:46.977028 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:48:46.976949    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5da24b7-3b69-483d-90af-01daac639565" containerName="cleanup"
Jul 13 19:48:46.979869 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:48:46.979851    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94"
Jul 13 19:48:46.982323 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:48:46.982300    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 19:48:46.982440 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:48:46.982301    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q5n9g\""
Jul 13 19:48:46.986497 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:48:46.986469    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94"]
Jul 13 19:48:47.123066 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:48:47.123020    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0d126f7-943b-43e2-80c0-3ff438e92ed2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94\" (UID: \"c0d126f7-943b-43e2-80c0-3ff438e92ed2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94"
Jul 13 19:48:47.223559 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:48:47.223522    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0d126f7-943b-43e2-80c0-3ff438e92ed2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94\" (UID: \"c0d126f7-943b-43e2-80c0-3ff438e92ed2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94"
Jul 13 19:48:47.224187 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:48:47.224162    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0d126f7-943b-43e2-80c0-3ff438e92ed2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94\" (UID: \"c0d126f7-943b-43e2-80c0-3ff438e92ed2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94"
Jul 13 19:48:47.291424 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:48:47.291337    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94"
Jul 13 19:48:47.433242 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:48:47.433081    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94"]
Jul 13 19:48:47.435592 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:48:47.435565    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0d126f7_943b_43e2_80c0_3ff438e92ed2.slice/crio-b7b37e5f2794b97afe80060fc33ed989c5355e6e13c739d39c31ee3a2d696d42 WatchSource:0}: Error finding container b7b37e5f2794b97afe80060fc33ed989c5355e6e13c739d39c31ee3a2d696d42: Status 404 returned error can't find the container with id b7b37e5f2794b97afe80060fc33ed989c5355e6e13c739d39c31ee3a2d696d42
Jul 13 19:48:47.471880 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:48:47.471843    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94" event={"ID":"c0d126f7-943b-43e2-80c0-3ff438e92ed2","Type":"ContainerStarted","Data":"b7b37e5f2794b97afe80060fc33ed989c5355e6e13c739d39c31ee3a2d696d42"}
Jul 13 19:48:48.477604 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:48:48.477566    2573 generic.go:356] "Generic (PLEG): container finished" podID="c0d126f7-943b-43e2-80c0-3ff438e92ed2" containerID="e13dfb3d07e6ee3ad94e7016b2c6701221a1c447095370f79985c46a02629cb5" exitCode=7
Jul 13 19:48:48.478075 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:48:48.477618    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94" event={"ID":"c0d126f7-943b-43e2-80c0-3ff438e92ed2","Type":"ContainerDied","Data":"e13dfb3d07e6ee3ad94e7016b2c6701221a1c447095370f79985c46a02629cb5"}
Jul 13 19:48:48.478075 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:48:48.477896    2573 scope.go:117] "RemoveContainer" containerID="e13dfb3d07e6ee3ad94e7016b2c6701221a1c447095370f79985c46a02629cb5"
Jul 13 19:48:49.484155 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:48:49.484062    2573 generic.go:356] "Generic (PLEG): container finished" podID="c0d126f7-943b-43e2-80c0-3ff438e92ed2" containerID="e083f1937f8a4ab1b825e6da85c5ad402f57f51b7107c1beda6711a6241af6ce" exitCode=7
Jul 13 19:48:49.484155 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:48:49.484127    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94" event={"ID":"c0d126f7-943b-43e2-80c0-3ff438e92ed2","Type":"ContainerDied","Data":"e083f1937f8a4ab1b825e6da85c5ad402f57f51b7107c1beda6711a6241af6ce"}
Jul 13 19:48:49.484603 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:48:49.484175    2573 scope.go:117] "RemoveContainer" containerID="e13dfb3d07e6ee3ad94e7016b2c6701221a1c447095370f79985c46a02629cb5"
Jul 13 19:48:49.484603 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:48:49.484380    2573 scope.go:117] "RemoveContainer" containerID="e083f1937f8a4ab1b825e6da85c5ad402f57f51b7107c1beda6711a6241af6ce"
Jul 13 19:48:49.484718 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:48:49.484617    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94_odh-ai-gateway-infra(c0d126f7-943b-43e2-80c0-3ff438e92ed2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94" podUID="c0d126f7-943b-43e2-80c0-3ff438e92ed2"
Jul 13 19:48:50.491203 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:48:50.491177    2573 scope.go:117] "RemoveContainer" containerID="e083f1937f8a4ab1b825e6da85c5ad402f57f51b7107c1beda6711a6241af6ce"
Jul 13 19:48:50.491696 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:48:50.491429    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94_odh-ai-gateway-infra(c0d126f7-943b-43e2-80c0-3ff438e92ed2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94" podUID="c0d126f7-943b-43e2-80c0-3ff438e92ed2"
Jul 13 19:49:05.042537 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:05.042502    2573 scope.go:117] "RemoveContainer" containerID="e083f1937f8a4ab1b825e6da85c5ad402f57f51b7107c1beda6711a6241af6ce"
Jul 13 19:49:05.569354 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:05.569323    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94" event={"ID":"c0d126f7-943b-43e2-80c0-3ff438e92ed2","Type":"ContainerStarted","Data":"f0dbd116ddc381379fbe46ef1922a46991093cd2d1b5c22695428382e5481c86"}
Jul 13 19:49:05.585601 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:05.585533    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94" podStartSLOduration=19.585516565 podStartE2EDuration="19.585516565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:48:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:49:05.584803091 +0000 UTC m=+1643.098679717" watchObservedRunningTime="2026-07-13 19:49:05.585516565 +0000 UTC m=+1643.099393170"
Jul 13 19:49:06.576429 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:06.576386    2573 generic.go:356] "Generic (PLEG): container finished" podID="c0d126f7-943b-43e2-80c0-3ff438e92ed2" containerID="f0dbd116ddc381379fbe46ef1922a46991093cd2d1b5c22695428382e5481c86" exitCode=0
Jul 13 19:49:06.576429 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:06.576420    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94" event={"ID":"c0d126f7-943b-43e2-80c0-3ff438e92ed2","Type":"ContainerDied","Data":"f0dbd116ddc381379fbe46ef1922a46991093cd2d1b5c22695428382e5481c86"}
Jul 13 19:49:06.576936 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:06.576485    2573 scope.go:117] "RemoveContainer" containerID="e083f1937f8a4ab1b825e6da85c5ad402f57f51b7107c1beda6711a6241af6ce"
Jul 13 19:49:06.596090 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:06.596059    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94"]
Jul 13 19:49:07.725553 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:07.725525    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94"
Jul 13 19:49:07.797886 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:07.797846    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0d126f7-943b-43e2-80c0-3ff438e92ed2-maas-api-service-ca\") pod \"c0d126f7-943b-43e2-80c0-3ff438e92ed2\" (UID: \"c0d126f7-943b-43e2-80c0-3ff438e92ed2\") "
Jul 13 19:49:07.798242 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:07.798216    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0d126f7-943b-43e2-80c0-3ff438e92ed2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c0d126f7-943b-43e2-80c0-3ff438e92ed2" (UID: "c0d126f7-943b-43e2-80c0-3ff438e92ed2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:49:07.898864 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:07.898765    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0d126f7-943b-43e2-80c0-3ff438e92ed2-maas-api-service-ca\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:49:08.588708 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:08.588677    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94"
Jul 13 19:49:08.588883 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:08.588684    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94" event={"ID":"c0d126f7-943b-43e2-80c0-3ff438e92ed2","Type":"ContainerDied","Data":"b7b37e5f2794b97afe80060fc33ed989c5355e6e13c739d39c31ee3a2d696d42"}
Jul 13 19:49:08.588883 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:08.588794    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7b37e5f2794b97afe80060fc33ed989c5355e6e13c739d39c31ee3a2d696d42"
Jul 13 19:49:08.616927 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:08.616890    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94"]
Jul 13 19:49:08.625839 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:08.625808    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-mpn94"]
Jul 13 19:49:09.048224 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:09.048189    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0d126f7-943b-43e2-80c0-3ff438e92ed2" path="/var/lib/kubelet/pods/c0d126f7-943b-43e2-80c0-3ff438e92ed2/volumes"
Jul 13 19:49:47.039939 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:47.039898    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-nctjc"]
Jul 13 19:49:47.040427 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:47.040237    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0d126f7-943b-43e2-80c0-3ff438e92ed2" containerName="revoke-keys"
Jul 13 19:49:47.040427 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:47.040249    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0d126f7-943b-43e2-80c0-3ff438e92ed2" containerName="revoke-keys"
Jul 13 19:49:47.040427 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:47.040258    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0d126f7-943b-43e2-80c0-3ff438e92ed2" containerName="revoke-keys"
Jul 13 19:49:47.040427 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:47.040264    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0d126f7-943b-43e2-80c0-3ff438e92ed2" containerName="revoke-keys"
Jul 13 19:49:47.040427 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:47.040273    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0d126f7-943b-43e2-80c0-3ff438e92ed2" containerName="revoke-keys"
Jul 13 19:49:47.040427 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:47.040279    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0d126f7-943b-43e2-80c0-3ff438e92ed2" containerName="revoke-keys"
Jul 13 19:49:47.040427 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:47.040340    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0d126f7-943b-43e2-80c0-3ff438e92ed2" containerName="revoke-keys"
Jul 13 19:49:47.040427 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:47.040347    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0d126f7-943b-43e2-80c0-3ff438e92ed2" containerName="revoke-keys"
Jul 13 19:49:47.043399 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:47.043378    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-nctjc"
Jul 13 19:49:47.046095 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:47.046068    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 19:49:47.046574 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:47.046552    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q5n9g\""
Jul 13 19:49:47.057384 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:47.057350    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-nctjc"]
Jul 13 19:49:47.138618 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:47.138582    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/288e10bd-518c-482d-92dd-5d655f887880-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1b0d941e-nctjc\" (UID: \"288e10bd-518c-482d-92dd-5d655f887880\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-nctjc"
Jul 13 19:49:47.239357 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:47.239320    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/288e10bd-518c-482d-92dd-5d655f887880-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1b0d941e-nctjc\" (UID: \"288e10bd-518c-482d-92dd-5d655f887880\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-nctjc"
Jul 13 19:49:47.239988 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:47.239964    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/288e10bd-518c-482d-92dd-5d655f887880-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1b0d941e-nctjc\" (UID: \"288e10bd-518c-482d-92dd-5d655f887880\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-nctjc"
Jul 13 19:49:47.355243 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:47.355148    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-nctjc"
Jul 13 19:49:47.492802 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:47.492774    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-nctjc"]
Jul 13 19:49:47.495489 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:49:47.495463    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod288e10bd_518c_482d_92dd_5d655f887880.slice/crio-a122d7ceefdbf1ec977a3f063c19fee6b8579828b002f02d7fd78039957094fd WatchSource:0}: Error finding container a122d7ceefdbf1ec977a3f063c19fee6b8579828b002f02d7fd78039957094fd: Status 404 returned error can't find the container with id a122d7ceefdbf1ec977a3f063c19fee6b8579828b002f02d7fd78039957094fd
Jul 13 19:49:47.775884 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:47.775847    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-nctjc" event={"ID":"288e10bd-518c-482d-92dd-5d655f887880","Type":"ContainerStarted","Data":"a122d7ceefdbf1ec977a3f063c19fee6b8579828b002f02d7fd78039957094fd"}
Jul 13 19:49:48.781922 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:48.781884    2573 generic.go:356] "Generic (PLEG): container finished" podID="288e10bd-518c-482d-92dd-5d655f887880" containerID="b80798d01bf0c48f2bb4b456bb80fd32af1c21e8980755443aaca48980b78807" exitCode=0
Jul 13 19:49:48.782341 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:48.781931    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-nctjc" event={"ID":"288e10bd-518c-482d-92dd-5d655f887880","Type":"ContainerDied","Data":"b80798d01bf0c48f2bb4b456bb80fd32af1c21e8980755443aaca48980b78807"}
Jul 13 19:49:49.928138 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:49.928109    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-nctjc"
Jul 13 19:49:49.958039 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:49.958001    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/288e10bd-518c-482d-92dd-5d655f887880-maas-api-service-ca\") pod \"288e10bd-518c-482d-92dd-5d655f887880\" (UID: \"288e10bd-518c-482d-92dd-5d655f887880\") "
Jul 13 19:49:49.958399 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:49.958373    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/288e10bd-518c-482d-92dd-5d655f887880-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "288e10bd-518c-482d-92dd-5d655f887880" (UID: "288e10bd-518c-482d-92dd-5d655f887880"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:49:50.059129 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:50.059038    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/288e10bd-518c-482d-92dd-5d655f887880-maas-api-service-ca\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:49:50.792340 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:50.792306    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-nctjc" event={"ID":"288e10bd-518c-482d-92dd-5d655f887880","Type":"ContainerDied","Data":"a122d7ceefdbf1ec977a3f063c19fee6b8579828b002f02d7fd78039957094fd"}
Jul 13 19:49:50.792340 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:50.792340    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1b0d941e-nctjc"
Jul 13 19:49:50.792539 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:49:50.792355    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a122d7ceefdbf1ec977a3f063c19fee6b8579828b002f02d7fd78039957094fd"
Jul 13 19:50:18.700316 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:18.700280    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p"]
Jul 13 19:50:18.700770 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:18.700650    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="288e10bd-518c-482d-92dd-5d655f887880" containerName="revoke-keys"
Jul 13 19:50:18.700770 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:18.700662    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="288e10bd-518c-482d-92dd-5d655f887880" containerName="revoke-keys"
Jul 13 19:50:18.700770 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:18.700726    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0d126f7-943b-43e2-80c0-3ff438e92ed2" containerName="revoke-keys"
Jul 13 19:50:18.700770 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:18.700733    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="288e10bd-518c-482d-92dd-5d655f887880" containerName="revoke-keys"
Jul 13 19:50:18.703629 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:18.703611    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p"
Jul 13 19:50:18.706152 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:18.706131    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 19:50:18.706267 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:18.706208    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q5n9g\""
Jul 13 19:50:18.711782 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:18.711754    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p"]
Jul 13 19:50:18.815807 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:18.815761    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p\" (UID: \"25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p"
Jul 13 19:50:18.917062 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:18.917023    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p\" (UID: \"25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p"
Jul 13 19:50:18.917728 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:18.917706    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p\" (UID: \"25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p"
Jul 13 19:50:19.014947 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:19.014837    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p"
Jul 13 19:50:19.158798 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:19.158762    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p"]
Jul 13 19:50:19.160210 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:50:19.160182    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25ebc55e_7f58_4cc8_ba2b_dfe4b3a793ac.slice/crio-54c203f2f40b2d96bd0d4ee6eb6dc0fb13ec4c5c37678b8da8a84fe5e2cf2eb7 WatchSource:0}: Error finding container 54c203f2f40b2d96bd0d4ee6eb6dc0fb13ec4c5c37678b8da8a84fe5e2cf2eb7: Status 404 returned error can't find the container with id 54c203f2f40b2d96bd0d4ee6eb6dc0fb13ec4c5c37678b8da8a84fe5e2cf2eb7
Jul 13 19:50:19.928951 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:19.928920    2573 generic.go:356] "Generic (PLEG): container finished" podID="25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac" containerID="fff30cf1026e2a3d60d1c257072d62630b25e68c5210d24a4d789ce15dcc52b2" exitCode=7
Jul 13 19:50:19.929347 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:19.928985    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p" event={"ID":"25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac","Type":"ContainerDied","Data":"fff30cf1026e2a3d60d1c257072d62630b25e68c5210d24a4d789ce15dcc52b2"}
Jul 13 19:50:19.929347 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:19.929024    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p" event={"ID":"25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac","Type":"ContainerStarted","Data":"54c203f2f40b2d96bd0d4ee6eb6dc0fb13ec4c5c37678b8da8a84fe5e2cf2eb7"}
Jul 13 19:50:19.929347 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:19.929177    2573 scope.go:117] "RemoveContainer" containerID="fff30cf1026e2a3d60d1c257072d62630b25e68c5210d24a4d789ce15dcc52b2"
Jul 13 19:50:20.935224 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:20.935131    2573 generic.go:356] "Generic (PLEG): container finished" podID="25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac" containerID="f0ba3869f39904822f2b1af8d3a72012dfbb5abc0e31723c9d806f1242d6217c" exitCode=7
Jul 13 19:50:20.935224 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:20.935214    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p" event={"ID":"25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac","Type":"ContainerDied","Data":"f0ba3869f39904822f2b1af8d3a72012dfbb5abc0e31723c9d806f1242d6217c"}
Jul 13 19:50:20.935704 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:20.935270    2573 scope.go:117] "RemoveContainer" containerID="fff30cf1026e2a3d60d1c257072d62630b25e68c5210d24a4d789ce15dcc52b2"
Jul 13 19:50:20.935704 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:20.935429    2573 scope.go:117] "RemoveContainer" containerID="f0ba3869f39904822f2b1af8d3a72012dfbb5abc0e31723c9d806f1242d6217c"
Jul 13 19:50:20.935704 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:50:20.935648    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p_odh-ai-gateway-infra(25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p" podUID="25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac"
Jul 13 19:50:21.942487 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:21.942460    2573 scope.go:117] "RemoveContainer" containerID="f0ba3869f39904822f2b1af8d3a72012dfbb5abc0e31723c9d806f1242d6217c"
Jul 13 19:50:21.942894 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:50:21.942726    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p_odh-ai-gateway-infra(25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p" podUID="25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac"
Jul 13 19:50:36.043418 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:36.043386    2573 scope.go:117] "RemoveContainer" containerID="f0ba3869f39904822f2b1af8d3a72012dfbb5abc0e31723c9d806f1242d6217c"
Jul 13 19:50:37.016311 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:37.016225    2573 generic.go:356] "Generic (PLEG): container finished" podID="25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac" containerID="7454457d22400fda813fc20c31adf4892a576868f25bf56b5a5e639bbaf4d5bc" exitCode=0
Jul 13 19:50:37.016311 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:37.016282    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p" event={"ID":"25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac","Type":"ContainerDied","Data":"7454457d22400fda813fc20c31adf4892a576868f25bf56b5a5e639bbaf4d5bc"}
Jul 13 19:50:37.016505 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:37.016331    2573 scope.go:117] "RemoveContainer" containerID="f0ba3869f39904822f2b1af8d3a72012dfbb5abc0e31723c9d806f1242d6217c"
Jul 13 19:50:38.049792 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:38.049738    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p"]
Jul 13 19:50:38.162788 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:38.162762    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p"
Jul 13 19:50:38.194152 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:38.194122    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac-maas-api-service-ca\") pod \"25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac\" (UID: \"25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac\") "
Jul 13 19:50:38.194492 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:38.194458    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac" (UID: "25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:50:38.295705 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:38.295609    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac-maas-api-service-ca\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:50:39.028409 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:39.028373    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p"
Jul 13 19:50:39.028593 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:39.028416    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p" event={"ID":"25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac","Type":"ContainerDied","Data":"54c203f2f40b2d96bd0d4ee6eb6dc0fb13ec4c5c37678b8da8a84fe5e2cf2eb7"}
Jul 13 19:50:39.028593 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:39.028502    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54c203f2f40b2d96bd0d4ee6eb6dc0fb13ec4c5c37678b8da8a84fe5e2cf2eb7"
Jul 13 19:50:39.054555 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:39.054518    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p"]
Jul 13 19:50:39.060294 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:39.060260    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-x4x6p"]
Jul 13 19:50:41.048078 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:50:41.048041    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac" path="/var/lib/kubelet/pods/25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac/volumes"
Jul 13 19:51:18.748915 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:18.748875    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-2r42d"]
Jul 13 19:51:18.749378 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:18.749228    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac" containerName="revoke-keys"
Jul 13 19:51:18.749378 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:18.749238    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac" containerName="revoke-keys"
Jul 13 19:51:18.749378 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:18.749250    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac" containerName="revoke-keys"
Jul 13 19:51:18.749378 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:18.749256    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac" containerName="revoke-keys"
Jul 13 19:51:18.749378 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:18.749323    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac" containerName="revoke-keys"
Jul 13 19:51:18.749378 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:18.749330    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac" containerName="revoke-keys"
Jul 13 19:51:18.752227 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:18.752208    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-2r42d"
Jul 13 19:51:18.754957 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:18.754933    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q5n9g\""
Jul 13 19:51:18.755078 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:18.754936    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 19:51:18.759652 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:18.759610    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-2r42d"]
Jul 13 19:51:18.858664 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:18.858615    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5f30572-8152-49a7-921c-d0c889b06dcf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f3b1e94a-2r42d\" (UID: \"a5f30572-8152-49a7-921c-d0c889b06dcf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-2r42d"
Jul 13 19:51:18.960166 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:18.960118    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5f30572-8152-49a7-921c-d0c889b06dcf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f3b1e94a-2r42d\" (UID: \"a5f30572-8152-49a7-921c-d0c889b06dcf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-2r42d"
Jul 13 19:51:18.960789 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:18.960768    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5f30572-8152-49a7-921c-d0c889b06dcf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f3b1e94a-2r42d\" (UID: \"a5f30572-8152-49a7-921c-d0c889b06dcf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-2r42d"
Jul 13 19:51:19.063755 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:19.063667    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-2r42d"
Jul 13 19:51:19.197050 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:19.197024    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-2r42d"]
Jul 13 19:51:19.198962 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:51:19.198935    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5f30572_8152_49a7_921c_d0c889b06dcf.slice/crio-e0202c986ba18c1ac8bc188cd28a29569c5c64db01ce790805af20ba2060c862 WatchSource:0}: Error finding container e0202c986ba18c1ac8bc188cd28a29569c5c64db01ce790805af20ba2060c862: Status 404 returned error can't find the container with id e0202c986ba18c1ac8bc188cd28a29569c5c64db01ce790805af20ba2060c862
Jul 13 19:51:19.221860 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:19.221831    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-2r42d" event={"ID":"a5f30572-8152-49a7-921c-d0c889b06dcf","Type":"ContainerStarted","Data":"e0202c986ba18c1ac8bc188cd28a29569c5c64db01ce790805af20ba2060c862"}
Jul 13 19:51:20.227960 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:20.227923    2573 generic.go:356] "Generic (PLEG): container finished" podID="a5f30572-8152-49a7-921c-d0c889b06dcf" containerID="54b6b52d4d047e0596ea6e9afee582e744b4fb75cfdff1b46728c00727aaf750" exitCode=0
Jul 13 19:51:20.228350 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:20.227996    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-2r42d" event={"ID":"a5f30572-8152-49a7-921c-d0c889b06dcf","Type":"ContainerDied","Data":"54b6b52d4d047e0596ea6e9afee582e744b4fb75cfdff1b46728c00727aaf750"}
Jul 13 19:51:21.367843 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:21.367814    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-2r42d"
Jul 13 19:51:21.484128 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:21.484025    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5f30572-8152-49a7-921c-d0c889b06dcf-maas-api-service-ca\") pod \"a5f30572-8152-49a7-921c-d0c889b06dcf\" (UID: \"a5f30572-8152-49a7-921c-d0c889b06dcf\") "
Jul 13 19:51:21.484439 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:21.484414    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5f30572-8152-49a7-921c-d0c889b06dcf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a5f30572-8152-49a7-921c-d0c889b06dcf" (UID: "a5f30572-8152-49a7-921c-d0c889b06dcf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:51:21.585329 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:21.585291    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5f30572-8152-49a7-921c-d0c889b06dcf-maas-api-service-ca\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:51:22.238217 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:22.238182    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-2r42d"
Jul 13 19:51:22.238217 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:22.238208    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f3b1e94a-2r42d" event={"ID":"a5f30572-8152-49a7-921c-d0c889b06dcf","Type":"ContainerDied","Data":"e0202c986ba18c1ac8bc188cd28a29569c5c64db01ce790805af20ba2060c862"}
Jul 13 19:51:22.238438 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:22.238264    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e0202c986ba18c1ac8bc188cd28a29569c5c64db01ce790805af20ba2060c862"
Jul 13 19:51:37.161022 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:37.160985    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-vrjgq"]
Jul 13 19:51:37.161402 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:37.161327    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac" containerName="revoke-keys"
Jul 13 19:51:37.161402 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:37.161338    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac" containerName="revoke-keys"
Jul 13 19:51:37.161402 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:37.161346    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5f30572-8152-49a7-921c-d0c889b06dcf" containerName="revoke-keys"
Jul 13 19:51:37.161402 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:37.161352    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5f30572-8152-49a7-921c-d0c889b06dcf" containerName="revoke-keys"
Jul 13 19:51:37.161530 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:37.161409    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5f30572-8152-49a7-921c-d0c889b06dcf" containerName="revoke-keys"
Jul 13 19:51:37.161530 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:37.161419    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="25ebc55e-7f58-4cc8-ba2b-dfe4b3a793ac" containerName="revoke-keys"
Jul 13 19:51:37.164475 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:37.164457    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-vrjgq"
Jul 13 19:51:37.167124 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:37.167097    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 19:51:37.167124 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:37.167112    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q5n9g\""
Jul 13 19:51:37.172661 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:37.172616    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-vrjgq"]
Jul 13 19:51:37.204319 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:37.204281    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eff1db10-d52e-465e-af96-285fbd0005a7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-32278165-vrjgq\" (UID: \"eff1db10-d52e-465e-af96-285fbd0005a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-vrjgq"
Jul 13 19:51:37.305601 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:37.305563    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eff1db10-d52e-465e-af96-285fbd0005a7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-32278165-vrjgq\" (UID: \"eff1db10-d52e-465e-af96-285fbd0005a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-vrjgq"
Jul 13 19:51:37.306177 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:37.306158    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eff1db10-d52e-465e-af96-285fbd0005a7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-32278165-vrjgq\" (UID: \"eff1db10-d52e-465e-af96-285fbd0005a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-vrjgq"
Jul 13 19:51:37.475975 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:37.475935    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-vrjgq"
Jul 13 19:51:37.613598 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:37.613570    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-vrjgq"]
Jul 13 19:51:37.615613 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:51:37.615579    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeff1db10_d52e_465e_af96_285fbd0005a7.slice/crio-640e6b04764036775a9c87a8f70d723c6716c5940a4c5e99169bcdc13541ecaf WatchSource:0}: Error finding container 640e6b04764036775a9c87a8f70d723c6716c5940a4c5e99169bcdc13541ecaf: Status 404 returned error can't find the container with id 640e6b04764036775a9c87a8f70d723c6716c5940a4c5e99169bcdc13541ecaf
Jul 13 19:51:38.316767 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:38.316736    2573 generic.go:356] "Generic (PLEG): container finished" podID="eff1db10-d52e-465e-af96-285fbd0005a7" containerID="9af9590670aab4bb22a98c11d10de707f3164192dc3b35fdd11580d4524f3f50" exitCode=7
Jul 13 19:51:38.317132 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:38.316787    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-vrjgq" event={"ID":"eff1db10-d52e-465e-af96-285fbd0005a7","Type":"ContainerDied","Data":"9af9590670aab4bb22a98c11d10de707f3164192dc3b35fdd11580d4524f3f50"}
Jul 13 19:51:38.317132 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:38.316841    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-vrjgq" event={"ID":"eff1db10-d52e-465e-af96-285fbd0005a7","Type":"ContainerStarted","Data":"640e6b04764036775a9c87a8f70d723c6716c5940a4c5e99169bcdc13541ecaf"}
Jul 13 19:51:38.317132 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:38.317019    2573 scope.go:117] "RemoveContainer" containerID="9af9590670aab4bb22a98c11d10de707f3164192dc3b35fdd11580d4524f3f50"
Jul 13 19:51:39.323039 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:39.323005    2573 generic.go:356] "Generic (PLEG): container finished" podID="eff1db10-d52e-465e-af96-285fbd0005a7" containerID="a8d0909403d78925b570b938b424ed6143e90300ecc04538bd88082b0fd47e3b" exitCode=7
Jul 13 19:51:39.323476 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:39.323092    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-vrjgq" event={"ID":"eff1db10-d52e-465e-af96-285fbd0005a7","Type":"ContainerDied","Data":"a8d0909403d78925b570b938b424ed6143e90300ecc04538bd88082b0fd47e3b"}
Jul 13 19:51:39.323476 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:39.323179    2573 scope.go:117] "RemoveContainer" containerID="9af9590670aab4bb22a98c11d10de707f3164192dc3b35fdd11580d4524f3f50"
Jul 13 19:51:39.323476 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:39.323337    2573 scope.go:117] "RemoveContainer" containerID="a8d0909403d78925b570b938b424ed6143e90300ecc04538bd88082b0fd47e3b"
Jul 13 19:51:39.323615 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:51:39.323572    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-32278165-vrjgq_odh-ai-gateway-infra(eff1db10-d52e-465e-af96-285fbd0005a7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-vrjgq" podUID="eff1db10-d52e-465e-af96-285fbd0005a7"
Jul 13 19:51:40.330003 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:40.329974    2573 scope.go:117] "RemoveContainer" containerID="a8d0909403d78925b570b938b424ed6143e90300ecc04538bd88082b0fd47e3b"
Jul 13 19:51:40.330401 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:51:40.330177    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-32278165-vrjgq_odh-ai-gateway-infra(eff1db10-d52e-465e-af96-285fbd0005a7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-vrjgq" podUID="eff1db10-d52e-465e-af96-285fbd0005a7"
Jul 13 19:51:43.006680 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:43.006653    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 19:51:43.009096 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:43.009073    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 19:51:43.010061 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:43.010041    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 19:51:43.012257 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:43.012240    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 19:51:54.043183 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:54.043149    2573 scope.go:117] "RemoveContainer" containerID="a8d0909403d78925b570b938b424ed6143e90300ecc04538bd88082b0fd47e3b"
Jul 13 19:51:54.397741 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:54.397704    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-vrjgq" event={"ID":"eff1db10-d52e-465e-af96-285fbd0005a7","Type":"ContainerStarted","Data":"c42c0edb088acea5fe1d43967d1232ec32884e73a72a389577ec3202a27e1409"}
Jul 13 19:51:54.416720 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:54.416665    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-vrjgq" podStartSLOduration=17.416649032 podStartE2EDuration="17.416649032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:51:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:51:54.414312507 +0000 UTC m=+1811.928189127" watchObservedRunningTime="2026-07-13 19:51:54.416649032 +0000 UTC m=+1811.930525632"
Jul 13 19:51:55.404367 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:55.404326    2573 generic.go:356] "Generic (PLEG): container finished" podID="eff1db10-d52e-465e-af96-285fbd0005a7" containerID="c42c0edb088acea5fe1d43967d1232ec32884e73a72a389577ec3202a27e1409" exitCode=0
Jul 13 19:51:55.404844 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:55.404388    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-vrjgq" event={"ID":"eff1db10-d52e-465e-af96-285fbd0005a7","Type":"ContainerDied","Data":"c42c0edb088acea5fe1d43967d1232ec32884e73a72a389577ec3202a27e1409"}
Jul 13 19:51:55.404844 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:55.404438    2573 scope.go:117] "RemoveContainer" containerID="a8d0909403d78925b570b938b424ed6143e90300ecc04538bd88082b0fd47e3b"
Jul 13 19:51:55.430309 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:55.430278    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-vrjgq"]
Jul 13 19:51:56.544363 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:56.544335    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-vrjgq"
Jul 13 19:51:56.578772 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:56.578738    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eff1db10-d52e-465e-af96-285fbd0005a7-maas-api-service-ca\") pod \"eff1db10-d52e-465e-af96-285fbd0005a7\" (UID: \"eff1db10-d52e-465e-af96-285fbd0005a7\") "
Jul 13 19:51:56.579102 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:56.579080    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eff1db10-d52e-465e-af96-285fbd0005a7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eff1db10-d52e-465e-af96-285fbd0005a7" (UID: "eff1db10-d52e-465e-af96-285fbd0005a7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:51:56.680210 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:56.680102    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eff1db10-d52e-465e-af96-285fbd0005a7-maas-api-service-ca\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:51:57.416348 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:57.416259    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-vrjgq"
Jul 13 19:51:57.416348 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:57.416295    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-vrjgq" event={"ID":"eff1db10-d52e-465e-af96-285fbd0005a7","Type":"ContainerDied","Data":"640e6b04764036775a9c87a8f70d723c6716c5940a4c5e99169bcdc13541ecaf"}
Jul 13 19:51:57.416348 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:57.416337    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="640e6b04764036775a9c87a8f70d723c6716c5940a4c5e99169bcdc13541ecaf"
Jul 13 19:51:57.441825 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:57.441789    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-vrjgq"]
Jul 13 19:51:57.449507 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:57.449475    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-vrjgq"]
Jul 13 19:51:59.047740 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:51:59.047704    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eff1db10-d52e-465e-af96-285fbd0005a7" path="/var/lib/kubelet/pods/eff1db10-d52e-465e-af96-285fbd0005a7/volumes"
Jul 13 19:52:37.222624 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:37.222565    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-q6lfr"]
Jul 13 19:52:37.223379 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:37.223352    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eff1db10-d52e-465e-af96-285fbd0005a7" containerName="revoke-keys"
Jul 13 19:52:37.223502 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:37.223390    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="eff1db10-d52e-465e-af96-285fbd0005a7" containerName="revoke-keys"
Jul 13 19:52:37.223502 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:37.223413    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eff1db10-d52e-465e-af96-285fbd0005a7" containerName="revoke-keys"
Jul 13 19:52:37.223502 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:37.223422    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="eff1db10-d52e-465e-af96-285fbd0005a7" containerName="revoke-keys"
Jul 13 19:52:37.223502 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:37.223447    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eff1db10-d52e-465e-af96-285fbd0005a7" containerName="revoke-keys"
Jul 13 19:52:37.223502 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:37.223456    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="eff1db10-d52e-465e-af96-285fbd0005a7" containerName="revoke-keys"
Jul 13 19:52:37.224089 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:37.223646    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="eff1db10-d52e-465e-af96-285fbd0005a7" containerName="revoke-keys"
Jul 13 19:52:37.224089 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:37.223667    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="eff1db10-d52e-465e-af96-285fbd0005a7" containerName="revoke-keys"
Jul 13 19:52:37.224089 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:37.223679    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="eff1db10-d52e-465e-af96-285fbd0005a7" containerName="revoke-keys"
Jul 13 19:52:37.227582 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:37.227561    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-q6lfr"
Jul 13 19:52:37.230445 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:37.230421    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 19:52:37.230659 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:37.230421    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q5n9g\""
Jul 13 19:52:37.231997 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:37.231968    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-q6lfr"]
Jul 13 19:52:37.341323 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:37.341282    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54b09556-56e8-429f-b17f-b69d24c45583-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-32278165-q6lfr\" (UID: \"54b09556-56e8-429f-b17f-b69d24c45583\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-q6lfr"
Jul 13 19:52:37.442072 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:37.442038    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54b09556-56e8-429f-b17f-b69d24c45583-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-32278165-q6lfr\" (UID: \"54b09556-56e8-429f-b17f-b69d24c45583\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-q6lfr"
Jul 13 19:52:37.442722 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:37.442703    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54b09556-56e8-429f-b17f-b69d24c45583-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-32278165-q6lfr\" (UID: \"54b09556-56e8-429f-b17f-b69d24c45583\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-q6lfr"
Jul 13 19:52:37.539131 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:37.539032    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-q6lfr"
Jul 13 19:52:37.672986 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:37.672960    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-q6lfr"]
Jul 13 19:52:37.674908 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:52:37.674877    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54b09556_56e8_429f_b17f_b69d24c45583.slice/crio-908e2de58244144fe97da8e87e3a5d761e3c43d4a045eaf2e5896702a5522ace WatchSource:0}: Error finding container 908e2de58244144fe97da8e87e3a5d761e3c43d4a045eaf2e5896702a5522ace: Status 404 returned error can't find the container with id 908e2de58244144fe97da8e87e3a5d761e3c43d4a045eaf2e5896702a5522ace
Jul 13 19:52:38.619348 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:38.619253    2573 generic.go:356] "Generic (PLEG): container finished" podID="54b09556-56e8-429f-b17f-b69d24c45583" containerID="29f6ac5a125e3e08915a0aa77741cd70c0dc8863058f3f5ee083f1870523f141" exitCode=0
Jul 13 19:52:38.619348 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:38.619336    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-q6lfr" event={"ID":"54b09556-56e8-429f-b17f-b69d24c45583","Type":"ContainerDied","Data":"29f6ac5a125e3e08915a0aa77741cd70c0dc8863058f3f5ee083f1870523f141"}
Jul 13 19:52:38.619782 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:38.619388    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-q6lfr" event={"ID":"54b09556-56e8-429f-b17f-b69d24c45583","Type":"ContainerStarted","Data":"908e2de58244144fe97da8e87e3a5d761e3c43d4a045eaf2e5896702a5522ace"}
Jul 13 19:52:39.758210 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:39.758180    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-q6lfr"
Jul 13 19:52:39.863592 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:39.863553    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54b09556-56e8-429f-b17f-b69d24c45583-maas-api-service-ca\") pod \"54b09556-56e8-429f-b17f-b69d24c45583\" (UID: \"54b09556-56e8-429f-b17f-b69d24c45583\") "
Jul 13 19:52:39.863953 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:39.863929    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54b09556-56e8-429f-b17f-b69d24c45583-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "54b09556-56e8-429f-b17f-b69d24c45583" (UID: "54b09556-56e8-429f-b17f-b69d24c45583"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:52:39.964377 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:39.964345    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54b09556-56e8-429f-b17f-b69d24c45583-maas-api-service-ca\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:52:40.630337 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:40.630301    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-q6lfr" event={"ID":"54b09556-56e8-429f-b17f-b69d24c45583","Type":"ContainerDied","Data":"908e2de58244144fe97da8e87e3a5d761e3c43d4a045eaf2e5896702a5522ace"}
Jul 13 19:52:40.630508 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:40.630360    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="908e2de58244144fe97da8e87e3a5d761e3c43d4a045eaf2e5896702a5522ace"
Jul 13 19:52:40.630508 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:52:40.630318    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32278165-q6lfr"
Jul 13 19:53:09.191401 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:09.191364    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8"]
Jul 13 19:53:09.191833 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:09.191722    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54b09556-56e8-429f-b17f-b69d24c45583" containerName="revoke-keys"
Jul 13 19:53:09.191833 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:09.191734    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="54b09556-56e8-429f-b17f-b69d24c45583" containerName="revoke-keys"
Jul 13 19:53:09.191833 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:09.191798    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="54b09556-56e8-429f-b17f-b69d24c45583" containerName="revoke-keys"
Jul 13 19:53:09.194878 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:09.194856    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8"
Jul 13 19:53:09.197440 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:09.197417    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q5n9g\""
Jul 13 19:53:09.197554 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:09.197446    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 19:53:09.201884 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:09.201844    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8"]
Jul 13 19:53:09.325709 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:09.325669    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a25ab333-a427-40ca-94c3-0dcb45fadfab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8\" (UID: \"a25ab333-a427-40ca-94c3-0dcb45fadfab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8"
Jul 13 19:53:09.426972 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:09.426925    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a25ab333-a427-40ca-94c3-0dcb45fadfab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8\" (UID: \"a25ab333-a427-40ca-94c3-0dcb45fadfab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8"
Jul 13 19:53:09.427583 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:09.427562    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a25ab333-a427-40ca-94c3-0dcb45fadfab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8\" (UID: \"a25ab333-a427-40ca-94c3-0dcb45fadfab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8"
Jul 13 19:53:09.506933 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:09.506834    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8"
Jul 13 19:53:09.640396 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:09.640368    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8"]
Jul 13 19:53:09.642715 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:53:09.642682    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25ab333_a427_40ca_94c3_0dcb45fadfab.slice/crio-dc3bd0eaddfcfeb05c5b6eb5df7f08978c9ff961c772ffbc3764376b050d6a36 WatchSource:0}: Error finding container dc3bd0eaddfcfeb05c5b6eb5df7f08978c9ff961c772ffbc3764376b050d6a36: Status 404 returned error can't find the container with id dc3bd0eaddfcfeb05c5b6eb5df7f08978c9ff961c772ffbc3764376b050d6a36
Jul 13 19:53:09.775949 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:09.775859    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8" event={"ID":"a25ab333-a427-40ca-94c3-0dcb45fadfab","Type":"ContainerStarted","Data":"dc3bd0eaddfcfeb05c5b6eb5df7f08978c9ff961c772ffbc3764376b050d6a36"}
Jul 13 19:53:10.782804 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:10.782765    2573 generic.go:356] "Generic (PLEG): container finished" podID="a25ab333-a427-40ca-94c3-0dcb45fadfab" containerID="47184b9b2c9478fcc50c6e47e41242f3e70f5304443dc66bf0820569690692ca" exitCode=7
Jul 13 19:53:10.783211 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:10.782854    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8" event={"ID":"a25ab333-a427-40ca-94c3-0dcb45fadfab","Type":"ContainerDied","Data":"47184b9b2c9478fcc50c6e47e41242f3e70f5304443dc66bf0820569690692ca"}
Jul 13 19:53:10.783211 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:10.783024    2573 scope.go:117] "RemoveContainer" containerID="47184b9b2c9478fcc50c6e47e41242f3e70f5304443dc66bf0820569690692ca"
Jul 13 19:53:11.789876 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:11.789839    2573 generic.go:356] "Generic (PLEG): container finished" podID="a25ab333-a427-40ca-94c3-0dcb45fadfab" containerID="3684b0c5ee031226a35e2f5bdfc808417dd1755e993439b924cc945da3c3a2d7" exitCode=7
Jul 13 19:53:11.790290 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:11.789900    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8" event={"ID":"a25ab333-a427-40ca-94c3-0dcb45fadfab","Type":"ContainerDied","Data":"3684b0c5ee031226a35e2f5bdfc808417dd1755e993439b924cc945da3c3a2d7"}
Jul 13 19:53:11.790290 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:11.789960    2573 scope.go:117] "RemoveContainer" containerID="47184b9b2c9478fcc50c6e47e41242f3e70f5304443dc66bf0820569690692ca"
Jul 13 19:53:11.790290 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:11.790140    2573 scope.go:117] "RemoveContainer" containerID="3684b0c5ee031226a35e2f5bdfc808417dd1755e993439b924cc945da3c3a2d7"
Jul 13 19:53:11.790406 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:53:11.790355    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8_odh-ai-gateway-infra(a25ab333-a427-40ca-94c3-0dcb45fadfab)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8" podUID="a25ab333-a427-40ca-94c3-0dcb45fadfab"
Jul 13 19:53:12.796391 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:12.796356    2573 scope.go:117] "RemoveContainer" containerID="3684b0c5ee031226a35e2f5bdfc808417dd1755e993439b924cc945da3c3a2d7"
Jul 13 19:53:12.796787 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:53:12.796557    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8_odh-ai-gateway-infra(a25ab333-a427-40ca-94c3-0dcb45fadfab)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8" podUID="a25ab333-a427-40ca-94c3-0dcb45fadfab"
Jul 13 19:53:27.042849 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:27.042818    2573 scope.go:117] "RemoveContainer" containerID="3684b0c5ee031226a35e2f5bdfc808417dd1755e993439b924cc945da3c3a2d7"
Jul 13 19:53:27.877689 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:27.877588    2573 generic.go:356] "Generic (PLEG): container finished" podID="a25ab333-a427-40ca-94c3-0dcb45fadfab" containerID="51fd4c791aafc74093967da27a710cc485b2c33c4605fcd7c40a00042c9b94be" exitCode=0
Jul 13 19:53:27.877689 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:27.877666    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8" event={"ID":"a25ab333-a427-40ca-94c3-0dcb45fadfab","Type":"ContainerDied","Data":"51fd4c791aafc74093967da27a710cc485b2c33c4605fcd7c40a00042c9b94be"}
Jul 13 19:53:27.877872 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:27.877732    2573 scope.go:117] "RemoveContainer" containerID="3684b0c5ee031226a35e2f5bdfc808417dd1755e993439b924cc945da3c3a2d7"
Jul 13 19:53:28.910009 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:28.909977    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8"]
Jul 13 19:53:29.021065 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:29.021040    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8"
Jul 13 19:53:29.107202 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:29.107162    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a25ab333-a427-40ca-94c3-0dcb45fadfab-maas-api-service-ca\") pod \"a25ab333-a427-40ca-94c3-0dcb45fadfab\" (UID: \"a25ab333-a427-40ca-94c3-0dcb45fadfab\") "
Jul 13 19:53:29.107578 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:29.107555    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a25ab333-a427-40ca-94c3-0dcb45fadfab-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a25ab333-a427-40ca-94c3-0dcb45fadfab" (UID: "a25ab333-a427-40ca-94c3-0dcb45fadfab"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:53:29.207932 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:29.207900    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a25ab333-a427-40ca-94c3-0dcb45fadfab-maas-api-service-ca\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:53:29.890691 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:29.890662    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8"
Jul 13 19:53:29.890882 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:29.890664    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8" event={"ID":"a25ab333-a427-40ca-94c3-0dcb45fadfab","Type":"ContainerDied","Data":"dc3bd0eaddfcfeb05c5b6eb5df7f08978c9ff961c772ffbc3764376b050d6a36"}
Jul 13 19:53:29.890882 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:29.890785    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc3bd0eaddfcfeb05c5b6eb5df7f08978c9ff961c772ffbc3764376b050d6a36"
Jul 13 19:53:29.923205 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:29.923154    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8"]
Jul 13 19:53:29.928599 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:29.928572    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-2lgd8"]
Jul 13 19:53:31.047653 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:53:31.047612    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a25ab333-a427-40ca-94c3-0dcb45fadfab" path="/var/lib/kubelet/pods/a25ab333-a427-40ca-94c3-0dcb45fadfab/volumes"
Jul 13 19:54:09.243733 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:09.243698    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-wrrpl"]
Jul 13 19:54:09.244174 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:09.244062    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a25ab333-a427-40ca-94c3-0dcb45fadfab" containerName="revoke-keys"
Jul 13 19:54:09.244174 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:09.244076    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a25ab333-a427-40ca-94c3-0dcb45fadfab" containerName="revoke-keys"
Jul 13 19:54:09.244174 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:09.244096    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a25ab333-a427-40ca-94c3-0dcb45fadfab" containerName="revoke-keys"
Jul 13 19:54:09.244174 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:09.244103    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a25ab333-a427-40ca-94c3-0dcb45fadfab" containerName="revoke-keys"
Jul 13 19:54:09.244174 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:09.244160    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a25ab333-a427-40ca-94c3-0dcb45fadfab" containerName="revoke-keys"
Jul 13 19:54:09.244174 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:09.244170    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a25ab333-a427-40ca-94c3-0dcb45fadfab" containerName="revoke-keys"
Jul 13 19:54:09.247342 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:09.247325    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-wrrpl"
Jul 13 19:54:09.249904 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:09.249880    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 19:54:09.250029 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:09.249908    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q5n9g\""
Jul 13 19:54:09.255874 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:09.255842    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-wrrpl"]
Jul 13 19:54:09.279950 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:09.279912    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0853e99-6cb4-4861-9888-511d92bc2c17-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7a28704c-wrrpl\" (UID: \"a0853e99-6cb4-4861-9888-511d92bc2c17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-wrrpl"
Jul 13 19:54:09.381126 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:09.381087    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0853e99-6cb4-4861-9888-511d92bc2c17-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7a28704c-wrrpl\" (UID: \"a0853e99-6cb4-4861-9888-511d92bc2c17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-wrrpl"
Jul 13 19:54:09.381760 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:09.381721    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0853e99-6cb4-4861-9888-511d92bc2c17-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7a28704c-wrrpl\" (UID: \"a0853e99-6cb4-4861-9888-511d92bc2c17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-wrrpl"
Jul 13 19:54:09.559109 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:09.559018    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-wrrpl"
Jul 13 19:54:09.693173 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:09.693146    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-wrrpl"]
Jul 13 19:54:09.695402 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:54:09.695365    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0853e99_6cb4_4861_9888_511d92bc2c17.slice/crio-0dbfc4ea573c003198b73d819945cc310e750b546829b8e545ed3bcbd48c6dda WatchSource:0}: Error finding container 0dbfc4ea573c003198b73d819945cc310e750b546829b8e545ed3bcbd48c6dda: Status 404 returned error can't find the container with id 0dbfc4ea573c003198b73d819945cc310e750b546829b8e545ed3bcbd48c6dda
Jul 13 19:54:10.094691 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:10.094655    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-wrrpl" event={"ID":"a0853e99-6cb4-4861-9888-511d92bc2c17","Type":"ContainerStarted","Data":"cd1b12c804393d2a38dfb1f5d646e27cf00e5e490fe1551fad4c9372063b37a3"}
Jul 13 19:54:10.094872 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:10.094708    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-wrrpl" event={"ID":"a0853e99-6cb4-4861-9888-511d92bc2c17","Type":"ContainerStarted","Data":"0dbfc4ea573c003198b73d819945cc310e750b546829b8e545ed3bcbd48c6dda"}
Jul 13 19:54:10.109337 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:10.109275    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-wrrpl" podStartSLOduration=1.109259509 podStartE2EDuration="1.109259509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:54:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:54:10.107051698 +0000 UTC m=+1947.620928307" watchObservedRunningTime="2026-07-13 19:54:10.109259509 +0000 UTC m=+1947.623136115"
Jul 13 19:54:11.101327 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:11.101290    2573 generic.go:356] "Generic (PLEG): container finished" podID="a0853e99-6cb4-4861-9888-511d92bc2c17" containerID="cd1b12c804393d2a38dfb1f5d646e27cf00e5e490fe1551fad4c9372063b37a3" exitCode=0
Jul 13 19:54:11.101749 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:11.101369    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-wrrpl" event={"ID":"a0853e99-6cb4-4861-9888-511d92bc2c17","Type":"ContainerDied","Data":"cd1b12c804393d2a38dfb1f5d646e27cf00e5e490fe1551fad4c9372063b37a3"}
Jul 13 19:54:12.235267 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:12.235243    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-wrrpl"
Jul 13 19:54:12.310182 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:12.310144    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0853e99-6cb4-4861-9888-511d92bc2c17-maas-api-service-ca\") pod \"a0853e99-6cb4-4861-9888-511d92bc2c17\" (UID: \"a0853e99-6cb4-4861-9888-511d92bc2c17\") "
Jul 13 19:54:12.310549 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:12.310524    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0853e99-6cb4-4861-9888-511d92bc2c17-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a0853e99-6cb4-4861-9888-511d92bc2c17" (UID: "a0853e99-6cb4-4861-9888-511d92bc2c17"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:54:12.411480 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:12.411390    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0853e99-6cb4-4861-9888-511d92bc2c17-maas-api-service-ca\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:54:13.112618 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:13.112588    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-wrrpl"
Jul 13 19:54:13.112807 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:13.112630    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7a28704c-wrrpl" event={"ID":"a0853e99-6cb4-4861-9888-511d92bc2c17","Type":"ContainerDied","Data":"0dbfc4ea573c003198b73d819945cc310e750b546829b8e545ed3bcbd48c6dda"}
Jul 13 19:54:13.112807 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:54:13.112708    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0dbfc4ea573c003198b73d819945cc310e750b546829b8e545ed3bcbd48c6dda"
Jul 13 19:55:43.118989 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:55:43.118953    2573 scope.go:117] "RemoveContainer" containerID="f0dbd116ddc381379fbe46ef1922a46991093cd2d1b5c22695428382e5481c86"
Jul 13 19:56:26.657949 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:26.657913    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ccb43b5e-5swpj"]
Jul 13 19:56:26.658400 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:26.658258    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a25ab333-a427-40ca-94c3-0dcb45fadfab" containerName="revoke-keys"
Jul 13 19:56:26.658400 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:26.658269    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a25ab333-a427-40ca-94c3-0dcb45fadfab" containerName="revoke-keys"
Jul 13 19:56:26.658400 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:26.658276    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0853e99-6cb4-4861-9888-511d92bc2c17" containerName="revoke-keys"
Jul 13 19:56:26.658400 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:26.658282    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0853e99-6cb4-4861-9888-511d92bc2c17" containerName="revoke-keys"
Jul 13 19:56:26.658400 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:26.658354    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0853e99-6cb4-4861-9888-511d92bc2c17" containerName="revoke-keys"
Jul 13 19:56:26.658400 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:26.658365    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a25ab333-a427-40ca-94c3-0dcb45fadfab" containerName="revoke-keys"
Jul 13 19:56:26.661562 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:26.661544    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ccb43b5e-5swpj"
Jul 13 19:56:26.664128 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:26.664108    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q5n9g\""
Jul 13 19:56:26.664248 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:26.664108    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 19:56:26.668484 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:26.668453    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ccb43b5e-5swpj"]
Jul 13 19:56:26.808523 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:26.808488    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da309781-4f9f-405e-b2d1-63cfa6b17284-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ccb43b5e-5swpj\" (UID: \"da309781-4f9f-405e-b2d1-63cfa6b17284\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ccb43b5e-5swpj"
Jul 13 19:56:26.910117 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:26.910018    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da309781-4f9f-405e-b2d1-63cfa6b17284-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ccb43b5e-5swpj\" (UID: \"da309781-4f9f-405e-b2d1-63cfa6b17284\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ccb43b5e-5swpj"
Jul 13 19:56:26.910741 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:26.910720    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da309781-4f9f-405e-b2d1-63cfa6b17284-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ccb43b5e-5swpj\" (UID: \"da309781-4f9f-405e-b2d1-63cfa6b17284\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ccb43b5e-5swpj"
Jul 13 19:56:26.973060 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:26.973028    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ccb43b5e-5swpj"
Jul 13 19:56:27.112504 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:27.112477    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ccb43b5e-5swpj"]
Jul 13 19:56:27.114743 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:56:27.114706    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda309781_4f9f_405e_b2d1_63cfa6b17284.slice/crio-ec295c8632df2d75450c9c0a7a9daafc7ec1a90113ca44b237482a0363571760 WatchSource:0}: Error finding container ec295c8632df2d75450c9c0a7a9daafc7ec1a90113ca44b237482a0363571760: Status 404 returned error can't find the container with id ec295c8632df2d75450c9c0a7a9daafc7ec1a90113ca44b237482a0363571760
Jul 13 19:56:27.775080 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:27.775034    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ccb43b5e-5swpj" event={"ID":"da309781-4f9f-405e-b2d1-63cfa6b17284","Type":"ContainerStarted","Data":"44f18d361af3c239be00786e6cf8367ef67bc48b2e4ce752f0f19c0b36c34385"}
Jul 13 19:56:27.775468 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:27.775096    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ccb43b5e-5swpj" event={"ID":"da309781-4f9f-405e-b2d1-63cfa6b17284","Type":"ContainerStarted","Data":"ec295c8632df2d75450c9c0a7a9daafc7ec1a90113ca44b237482a0363571760"}
Jul 13 19:56:27.791655 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:27.791579    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ccb43b5e-5swpj" podStartSLOduration=1.7915626919999998 podStartE2EDuration="1.791562692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:56:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:56:27.789737419 +0000 UTC m=+2085.303614026" watchObservedRunningTime="2026-07-13 19:56:27.791562692 +0000 UTC m=+2085.305439298"
Jul 13 19:56:28.781955 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:28.781922    2573 generic.go:356] "Generic (PLEG): container finished" podID="da309781-4f9f-405e-b2d1-63cfa6b17284" containerID="44f18d361af3c239be00786e6cf8367ef67bc48b2e4ce752f0f19c0b36c34385" exitCode=0
Jul 13 19:56:28.782347 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:28.781999    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ccb43b5e-5swpj" event={"ID":"da309781-4f9f-405e-b2d1-63cfa6b17284","Type":"ContainerDied","Data":"44f18d361af3c239be00786e6cf8367ef67bc48b2e4ce752f0f19c0b36c34385"}
Jul 13 19:56:29.922355 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:29.922328    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ccb43b5e-5swpj"
Jul 13 19:56:30.038990 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:30.038880    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da309781-4f9f-405e-b2d1-63cfa6b17284-maas-api-service-ca\") pod \"da309781-4f9f-405e-b2d1-63cfa6b17284\" (UID: \"da309781-4f9f-405e-b2d1-63cfa6b17284\") "
Jul 13 19:56:30.039297 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:30.039268    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da309781-4f9f-405e-b2d1-63cfa6b17284-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "da309781-4f9f-405e-b2d1-63cfa6b17284" (UID: "da309781-4f9f-405e-b2d1-63cfa6b17284"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:56:30.139683 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:30.139630    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da309781-4f9f-405e-b2d1-63cfa6b17284-maas-api-service-ca\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:56:30.792586 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:30.792551    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ccb43b5e-5swpj" event={"ID":"da309781-4f9f-405e-b2d1-63cfa6b17284","Type":"ContainerDied","Data":"ec295c8632df2d75450c9c0a7a9daafc7ec1a90113ca44b237482a0363571760"}
Jul 13 19:56:30.792783 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:30.792605    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec295c8632df2d75450c9c0a7a9daafc7ec1a90113ca44b237482a0363571760"
Jul 13 19:56:30.792783 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:30.792565    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ccb43b5e-5swpj"
Jul 13 19:56:43.037392 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:43.037362    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 19:56:43.039845 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:43.039822    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 19:56:43.041990 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:43.041967    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 19:56:43.044588 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:43.044569    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 19:56:43.138787 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:56:43.138756    2573 scope.go:117] "RemoveContainer" containerID="7454457d22400fda813fc20c31adf4892a576868f25bf56b5a5e639bbaf4d5bc"
Jul 13 19:57:29.195367 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.195324    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"]
Jul 13 19:57:29.195858 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.195842    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da309781-4f9f-405e-b2d1-63cfa6b17284" containerName="revoke-keys"
Jul 13 19:57:29.195909 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.195862    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="da309781-4f9f-405e-b2d1-63cfa6b17284" containerName="revoke-keys"
Jul 13 19:57:29.195988 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.195976    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="da309781-4f9f-405e-b2d1-63cfa6b17284" containerName="revoke-keys"
Jul 13 19:57:29.199415 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.199395    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:29.208881 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.208847    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-d67d76ca\"/\"e2e-lifecycle-model-d67d76ca-kserve-self-signed-certs\""
Jul 13 19:57:29.209048 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.208900    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-d67d76ca\"/\"default-dockercfg-b54bf\""
Jul 13 19:57:29.209893 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.209867    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-d67d76ca\"/\"kube-root-ca.crt\""
Jul 13 19:57:29.210035 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.209919    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-d67d76ca\"/\"openshift-service-ca.crt\""
Jul 13 19:57:29.211665 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.211626    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"]
Jul 13 19:57:29.250730 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.250700    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-tmp-dir\") pod \"e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") " pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:29.250730 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.250734    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b944177-bad4-448e-a776-011b71fecd70-tls-certs\") pod \"e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") " pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:29.250987 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.250755    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-dshm\") pod \"e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") " pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:29.250987 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.250897    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-home\") pod \"e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") " pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:29.250987 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.250935    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-model-cache\") pod \"e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") " pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:29.251115 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.251004    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lgbn\" (UniqueName: \"kubernetes.io/projected/3b944177-bad4-448e-a776-011b71fecd70-kube-api-access-4lgbn\") pod \"e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") " pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:29.351684 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.351613    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b944177-bad4-448e-a776-011b71fecd70-tls-certs\") pod \"e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") " pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:29.351869 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.351785    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-dshm\") pod \"e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") " pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:29.351941 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.351877    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-home\") pod \"e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") " pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:29.351941 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.351917    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-model-cache\") pod \"e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") " pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:29.352055 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.351988    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lgbn\" (UniqueName: \"kubernetes.io/projected/3b944177-bad4-448e-a776-011b71fecd70-kube-api-access-4lgbn\") pod \"e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") " pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:29.352055 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.352032    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-tmp-dir\") pod \"e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") " pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:29.352270 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.352243    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-home\") pod \"e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") " pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:29.352359 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.352307    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-model-cache\") pod \"e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") " pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:29.352420 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.352374    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-tmp-dir\") pod \"e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") " pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:29.354051 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.354022    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-dshm\") pod \"e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") " pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:29.354325 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.354305    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b944177-bad4-448e-a776-011b71fecd70-tls-certs\") pod \"e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") " pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:29.359558 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.359539    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lgbn\" (UniqueName: \"kubernetes.io/projected/3b944177-bad4-448e-a776-011b71fecd70-kube-api-access-4lgbn\") pod \"e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") " pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:29.519322 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.519225    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:29.653566 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:29.653509    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"]
Jul 13 19:57:29.656564 ip-10-0-143-91 kubenswrapper[2573]: W0713 19:57:29.656533    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b944177_bad4_448e_a776_011b71fecd70.slice/crio-1167add4baca718ae2e84e7caab1f81639500a190917246ed0cccacb8d0b1e7f WatchSource:0}: Error finding container 1167add4baca718ae2e84e7caab1f81639500a190917246ed0cccacb8d0b1e7f: Status 404 returned error can't find the container with id 1167add4baca718ae2e84e7caab1f81639500a190917246ed0cccacb8d0b1e7f
Jul 13 19:57:30.092415 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:30.092383    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl" event={"ID":"3b944177-bad4-448e-a776-011b71fecd70","Type":"ContainerStarted","Data":"a9f7dbd61ddbad82848884f8a78194f60544e0d89f4d33e43774d7c0b7dbfab3"}
Jul 13 19:57:30.092609 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:30.092424    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl" event={"ID":"3b944177-bad4-448e-a776-011b71fecd70","Type":"ContainerStarted","Data":"1167add4baca718ae2e84e7caab1f81639500a190917246ed0cccacb8d0b1e7f"}
Jul 13 19:57:30.109113 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:30.109059    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl" podStartSLOduration=1.109042428 podStartE2EDuration="1.109042428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:57:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:57:30.107380539 +0000 UTC m=+2147.621257146" watchObservedRunningTime="2026-07-13 19:57:30.109042428 +0000 UTC m=+2147.622919033"
Jul 13 19:57:39.520088 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:39.519985    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:39.520473 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:39.520077    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:39.532564 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:39.532537    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:57:40.152711 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:57:40.152668    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:58:24.075360 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:24.075320    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"]
Jul 13 19:58:39.095610 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.095567    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl" podUID="3b944177-bad4-448e-a776-011b71fecd70" containerName="main" containerID="cri-o://a9f7dbd61ddbad82848884f8a78194f60544e0d89f4d33e43774d7c0b7dbfab3" gracePeriod=45
Jul 13 19:58:39.348475 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.348416    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:58:39.430093 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.430062    2573 generic.go:356] "Generic (PLEG): container finished" podID="3b944177-bad4-448e-a776-011b71fecd70" containerID="a9f7dbd61ddbad82848884f8a78194f60544e0d89f4d33e43774d7c0b7dbfab3" exitCode=0
Jul 13 19:58:39.430264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.430107    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"
Jul 13 19:58:39.430264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.430156    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl" event={"ID":"3b944177-bad4-448e-a776-011b71fecd70","Type":"ContainerDied","Data":"a9f7dbd61ddbad82848884f8a78194f60544e0d89f4d33e43774d7c0b7dbfab3"}
Jul 13 19:58:39.430264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.430200    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl" event={"ID":"3b944177-bad4-448e-a776-011b71fecd70","Type":"ContainerDied","Data":"1167add4baca718ae2e84e7caab1f81639500a190917246ed0cccacb8d0b1e7f"}
Jul 13 19:58:39.430264 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.430220    2573 scope.go:117] "RemoveContainer" containerID="a9f7dbd61ddbad82848884f8a78194f60544e0d89f4d33e43774d7c0b7dbfab3"
Jul 13 19:58:39.440414 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.440393    2573 scope.go:117] "RemoveContainer" containerID="a9f7dbd61ddbad82848884f8a78194f60544e0d89f4d33e43774d7c0b7dbfab3"
Jul 13 19:58:39.440686 ip-10-0-143-91 kubenswrapper[2573]: E0713 19:58:39.440666    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9f7dbd61ddbad82848884f8a78194f60544e0d89f4d33e43774d7c0b7dbfab3\": container with ID starting with a9f7dbd61ddbad82848884f8a78194f60544e0d89f4d33e43774d7c0b7dbfab3 not found: ID does not exist" containerID="a9f7dbd61ddbad82848884f8a78194f60544e0d89f4d33e43774d7c0b7dbfab3"
Jul 13 19:58:39.440747 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.440695    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9f7dbd61ddbad82848884f8a78194f60544e0d89f4d33e43774d7c0b7dbfab3"} err="failed to get container status \"a9f7dbd61ddbad82848884f8a78194f60544e0d89f4d33e43774d7c0b7dbfab3\": rpc error: code = NotFound desc = could not find container \"a9f7dbd61ddbad82848884f8a78194f60544e0d89f4d33e43774d7c0b7dbfab3\": container with ID starting with a9f7dbd61ddbad82848884f8a78194f60544e0d89f4d33e43774d7c0b7dbfab3 not found: ID does not exist"
Jul 13 19:58:39.462944 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.462919    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b944177-bad4-448e-a776-011b71fecd70-tls-certs\") pod \"3b944177-bad4-448e-a776-011b71fecd70\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") "
Jul 13 19:58:39.463022 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.462963    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-tmp-dir\") pod \"3b944177-bad4-448e-a776-011b71fecd70\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") "
Jul 13 19:58:39.463022 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.462998    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-home\") pod \"3b944177-bad4-448e-a776-011b71fecd70\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") "
Jul 13 19:58:39.463133 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.463096    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lgbn\" (UniqueName: \"kubernetes.io/projected/3b944177-bad4-448e-a776-011b71fecd70-kube-api-access-4lgbn\") pod \"3b944177-bad4-448e-a776-011b71fecd70\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") "
Jul 13 19:58:39.463193 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.463164    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-model-cache\") pod \"3b944177-bad4-448e-a776-011b71fecd70\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") "
Jul 13 19:58:39.463245 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.463227    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-home" (OuterVolumeSpecName: "home") pod "3b944177-bad4-448e-a776-011b71fecd70" (UID: "3b944177-bad4-448e-a776-011b71fecd70"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:58:39.463300 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.463251    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-dshm\") pod \"3b944177-bad4-448e-a776-011b71fecd70\" (UID: \"3b944177-bad4-448e-a776-011b71fecd70\") "
Jul 13 19:58:39.463300 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.463256    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3b944177-bad4-448e-a776-011b71fecd70" (UID: "3b944177-bad4-448e-a776-011b71fecd70"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:58:39.463447 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.463421    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-model-cache" (OuterVolumeSpecName: "model-cache") pod "3b944177-bad4-448e-a776-011b71fecd70" (UID: "3b944177-bad4-448e-a776-011b71fecd70"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:58:39.463547 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.463535    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-home\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:58:39.463596 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.463554    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-model-cache\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:58:39.463596 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.463569    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-tmp-dir\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:58:39.465260 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.465234    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-dshm" (OuterVolumeSpecName: "dshm") pod "3b944177-bad4-448e-a776-011b71fecd70" (UID: "3b944177-bad4-448e-a776-011b71fecd70"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:58:39.465344 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.465260    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b944177-bad4-448e-a776-011b71fecd70-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3b944177-bad4-448e-a776-011b71fecd70" (UID: "3b944177-bad4-448e-a776-011b71fecd70"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:58:39.465385 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.465343    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b944177-bad4-448e-a776-011b71fecd70-kube-api-access-4lgbn" (OuterVolumeSpecName: "kube-api-access-4lgbn") pod "3b944177-bad4-448e-a776-011b71fecd70" (UID: "3b944177-bad4-448e-a776-011b71fecd70"). InnerVolumeSpecName "kube-api-access-4lgbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:58:39.564855 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.564826    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lgbn\" (UniqueName: \"kubernetes.io/projected/3b944177-bad4-448e-a776-011b71fecd70-kube-api-access-4lgbn\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:58:39.564855 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.564854    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b944177-bad4-448e-a776-011b71fecd70-dshm\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:58:39.565048 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.564870    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b944177-bad4-448e-a776-011b71fecd70-tls-certs\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 19:58:39.751221 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.751185    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"]
Jul 13 19:58:39.754840 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:39.754815    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-d67d76ca/e2e-lifecycle-model-d67d76ca-kserve-5586496d67-2szzl"]
Jul 13 19:58:41.048694 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:41.048657    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b944177-bad4-448e-a776-011b71fecd70" path="/var/lib/kubelet/pods/3b944177-bad4-448e-a776-011b71fecd70/volumes"
Jul 13 19:58:43.164070 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:58:43.164039    2573 scope.go:117] "RemoveContainer" containerID="c42c0edb088acea5fe1d43967d1232ec32884e73a72a389577ec3202a27e1409"
Jul 13 19:59:43.188500 ip-10-0-143-91 kubenswrapper[2573]: I0713 19:59:43.188469    2573 scope.go:117] "RemoveContainer" containerID="51fd4c791aafc74093967da27a710cc485b2c33c4605fcd7c40a00042c9b94be"
Jul 13 20:00:00.149080 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:00.149037    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732880-4fdvj"]
Jul 13 20:00:00.149552 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:00.149416    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b944177-bad4-448e-a776-011b71fecd70" containerName="main"
Jul 13 20:00:00.149552 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:00.149427    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b944177-bad4-448e-a776-011b71fecd70" containerName="main"
Jul 13 20:00:00.149552 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:00.149494    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b944177-bad4-448e-a776-011b71fecd70" containerName="main"
Jul 13 20:00:00.152601 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:00.152583    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732880-4fdvj"
Jul 13 20:00:00.155433 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:00.155412    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q5n9g\""
Jul 13 20:00:00.160744 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:00.160712    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732880-4fdvj"]
Jul 13 20:00:00.164244 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:00.164222    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732880-4fdvj"
Jul 13 20:00:00.302682 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:00.302659    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732880-4fdvj"]
Jul 13 20:00:00.305236 ip-10-0-143-91 kubenswrapper[2573]: W0713 20:00:00.305209    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8be5d623_00b3_46ba_bc0f_c2cc35065ba9.slice/crio-6f3b25b1a927dddc2c116507090f482c0aa493ab3ad3478f23e9e504a6487190 WatchSource:0}: Error finding container 6f3b25b1a927dddc2c116507090f482c0aa493ab3ad3478f23e9e504a6487190: Status 404 returned error can't find the container with id 6f3b25b1a927dddc2c116507090f482c0aa493ab3ad3478f23e9e504a6487190
Jul 13 20:00:00.829134 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:00.829038    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732880-4fdvj" event={"ID":"8be5d623-00b3-46ba-bc0f-c2cc35065ba9","Type":"ContainerStarted","Data":"ac3d00be438a5b48fe6902753ce5e6c5d3eab0d54ddca519ff1837bc63b6ca7a"}
Jul 13 20:00:00.829134 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:00.829100    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732880-4fdvj" event={"ID":"8be5d623-00b3-46ba-bc0f-c2cc35065ba9","Type":"ContainerStarted","Data":"6f3b25b1a927dddc2c116507090f482c0aa493ab3ad3478f23e9e504a6487190"}
Jul 13 20:00:00.847039 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:00.846985    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732880-4fdvj" podStartSLOduration=0.846969919 podStartE2EDuration="846.969919ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:00:00.845492027 +0000 UTC m=+2298.359368636" watchObservedRunningTime="2026-07-13 20:00:00.846969919 +0000 UTC m=+2298.360846525"
Jul 13 20:00:01.835259 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:01.835223    2573 generic.go:356] "Generic (PLEG): container finished" podID="8be5d623-00b3-46ba-bc0f-c2cc35065ba9" containerID="ac3d00be438a5b48fe6902753ce5e6c5d3eab0d54ddca519ff1837bc63b6ca7a" exitCode=0
Jul 13 20:00:01.835677 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:01.835306    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732880-4fdvj" event={"ID":"8be5d623-00b3-46ba-bc0f-c2cc35065ba9","Type":"ContainerDied","Data":"ac3d00be438a5b48fe6902753ce5e6c5d3eab0d54ddca519ff1837bc63b6ca7a"}
Jul 13 20:00:02.976422 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:02.976391    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732880-4fdvj"
Jul 13 20:00:03.845413 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:03.845379    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732880-4fdvj" event={"ID":"8be5d623-00b3-46ba-bc0f-c2cc35065ba9","Type":"ContainerDied","Data":"6f3b25b1a927dddc2c116507090f482c0aa493ab3ad3478f23e9e504a6487190"}
Jul 13 20:00:03.845413 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:03.845409    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732880-4fdvj"
Jul 13 20:00:03.845615 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:03.845431    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f3b25b1a927dddc2c116507090f482c0aa493ab3ad3478f23e9e504a6487190"
Jul 13 20:00:38.609365 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.609271    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"]
Jul 13 20:00:38.609971 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.609821    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8be5d623-00b3-46ba-bc0f-c2cc35065ba9" containerName="cleanup"
Jul 13 20:00:38.609971 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.609840    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8be5d623-00b3-46ba-bc0f-c2cc35065ba9" containerName="cleanup"
Jul 13 20:00:38.609971 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.609925    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8be5d623-00b3-46ba-bc0f-c2cc35065ba9" containerName="cleanup"
Jul 13 20:00:38.612846 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.612824    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:38.616747 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.616719    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-dffe11\"/\"kube-root-ca.crt\""
Jul 13 20:00:38.616879 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.616748    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-dffe11\"/\"test-model-dffe11-kserve-self-signed-certs\""
Jul 13 20:00:38.616879 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.616725    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-dffe11\"/\"default-dockercfg-hmkzs\""
Jul 13 20:00:38.616879 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.616796    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-dffe11\"/\"openshift-service-ca.crt\""
Jul 13 20:00:38.623010 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.622988    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"]
Jul 13 20:00:38.694006 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.693965    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-tmp-dir\") pod \"test-model-dffe11-kserve-6b9968d55f-wnd6d\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") " pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:38.694187 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.694024    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-dshm\") pod \"test-model-dffe11-kserve-6b9968d55f-wnd6d\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") " pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:38.694187 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.694051    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-tls-certs\") pod \"test-model-dffe11-kserve-6b9968d55f-wnd6d\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") " pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:38.694187 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.694089    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-model-cache\") pod \"test-model-dffe11-kserve-6b9968d55f-wnd6d\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") " pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:38.694187 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.694154    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxq76\" (UniqueName: \"kubernetes.io/projected/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-kube-api-access-fxq76\") pod \"test-model-dffe11-kserve-6b9968d55f-wnd6d\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") " pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:38.694187 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.694180    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-home\") pod \"test-model-dffe11-kserve-6b9968d55f-wnd6d\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") " pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:38.795580 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.795547    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-tmp-dir\") pod \"test-model-dffe11-kserve-6b9968d55f-wnd6d\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") " pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:38.795777 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.795593    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-dshm\") pod \"test-model-dffe11-kserve-6b9968d55f-wnd6d\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") " pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:38.795777 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.795610    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-tls-certs\") pod \"test-model-dffe11-kserve-6b9968d55f-wnd6d\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") " pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:38.795777 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.795659    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-model-cache\") pod \"test-model-dffe11-kserve-6b9968d55f-wnd6d\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") " pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:38.795777 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.795688    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxq76\" (UniqueName: \"kubernetes.io/projected/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-kube-api-access-fxq76\") pod \"test-model-dffe11-kserve-6b9968d55f-wnd6d\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") " pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:38.795777 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.795706    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-home\") pod \"test-model-dffe11-kserve-6b9968d55f-wnd6d\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") " pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:38.796100 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.795978    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-tmp-dir\") pod \"test-model-dffe11-kserve-6b9968d55f-wnd6d\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") " pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:38.796100 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.796071    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-home\") pod \"test-model-dffe11-kserve-6b9968d55f-wnd6d\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") " pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:38.796100 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.796080    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-model-cache\") pod \"test-model-dffe11-kserve-6b9968d55f-wnd6d\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") " pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:38.798001 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.797977    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-dshm\") pod \"test-model-dffe11-kserve-6b9968d55f-wnd6d\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") " pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:38.798094 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.798040    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-tls-certs\") pod \"test-model-dffe11-kserve-6b9968d55f-wnd6d\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") " pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:38.803912 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.803887    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxq76\" (UniqueName: \"kubernetes.io/projected/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-kube-api-access-fxq76\") pod \"test-model-dffe11-kserve-6b9968d55f-wnd6d\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") " pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:38.923189 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:38.923096    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:39.057085 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:39.057055    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"]
Jul 13 20:00:39.058433 ip-10-0-143-91 kubenswrapper[2573]: W0713 20:00:39.058395    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode610ab67_33e3_4dd0_aa38_fc5ac4835cef.slice/crio-9b60f3269c88a8f95ce915ecd29862241f0aaee314db903d7286adee0a1a9c82 WatchSource:0}: Error finding container 9b60f3269c88a8f95ce915ecd29862241f0aaee314db903d7286adee0a1a9c82: Status 404 returned error can't find the container with id 9b60f3269c88a8f95ce915ecd29862241f0aaee314db903d7286adee0a1a9c82
Jul 13 20:00:40.031750 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:40.031712    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d" event={"ID":"e610ab67-33e3-4dd0-aa38-fc5ac4835cef","Type":"ContainerStarted","Data":"3adf43a4b9df660e4e2cc185da2b94d6a2a11a0bacb74fed45175a1ab9a0d9ab"}
Jul 13 20:00:40.031750 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:40.031757    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d" event={"ID":"e610ab67-33e3-4dd0-aa38-fc5ac4835cef","Type":"ContainerStarted","Data":"9b60f3269c88a8f95ce915ecd29862241f0aaee314db903d7286adee0a1a9c82"}
Jul 13 20:00:40.048321 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:40.048268    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d" podStartSLOduration=2.048252805 podStartE2EDuration="2.048252805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:00:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:00:40.048025017 +0000 UTC m=+2337.561901645" watchObservedRunningTime="2026-07-13 20:00:40.048252805 +0000 UTC m=+2337.562129410"
Jul 13 20:00:48.923722 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:48.923679    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:48.925988 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:48.923809    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:48.936601 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:48.936580    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:00:49.083863 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:00:49.083832    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:01:02.798645 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:02.798595    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"]
Jul 13 20:01:02.802327 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:02.802310    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:02.806238 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:02.806184    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-7b8c26\"/\"test-model-7b8c26-kserve-self-signed-certs\""
Jul 13 20:01:02.806238 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:02.806238    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-7b8c26\"/\"default-dockercfg-75g26\""
Jul 13 20:01:02.806427 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:02.806260    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-7b8c26\"/\"kube-root-ca.crt\""
Jul 13 20:01:02.806554 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:02.806534    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-7b8c26\"/\"openshift-service-ca.crt\""
Jul 13 20:01:02.812014 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:02.811992    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"]
Jul 13 20:01:02.917216 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:02.917178    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-tmp-dir\") pod \"test-model-7b8c26-kserve-585b9cfb65-q6g9p\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") " pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:02.917405 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:02.917295    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-dshm\") pod \"test-model-7b8c26-kserve-585b9cfb65-q6g9p\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") " pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:02.917405 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:02.917333    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z5z8\" (UniqueName: \"kubernetes.io/projected/b162c536-4383-4027-a940-9d4c29ed6bca-kube-api-access-8z5z8\") pod \"test-model-7b8c26-kserve-585b9cfb65-q6g9p\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") " pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:02.917405 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:02.917374    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-home\") pod \"test-model-7b8c26-kserve-585b9cfb65-q6g9p\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") " pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:02.917405 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:02.917398    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-model-cache\") pod \"test-model-7b8c26-kserve-585b9cfb65-q6g9p\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") " pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:02.917677 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:02.917531    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b162c536-4383-4027-a940-9d4c29ed6bca-tls-certs\") pod \"test-model-7b8c26-kserve-585b9cfb65-q6g9p\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") " pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:03.018998 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:03.018949    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b162c536-4383-4027-a940-9d4c29ed6bca-tls-certs\") pod \"test-model-7b8c26-kserve-585b9cfb65-q6g9p\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") " pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:03.019205 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:03.019034    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-tmp-dir\") pod \"test-model-7b8c26-kserve-585b9cfb65-q6g9p\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") " pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:03.019205 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:03.019082    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-dshm\") pod \"test-model-7b8c26-kserve-585b9cfb65-q6g9p\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") " pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:03.019205 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:03.019109    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z5z8\" (UniqueName: \"kubernetes.io/projected/b162c536-4383-4027-a940-9d4c29ed6bca-kube-api-access-8z5z8\") pod \"test-model-7b8c26-kserve-585b9cfb65-q6g9p\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") " pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:03.019205 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:03.019146    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-home\") pod \"test-model-7b8c26-kserve-585b9cfb65-q6g9p\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") " pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:03.019205 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:03.019165    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-model-cache\") pod \"test-model-7b8c26-kserve-585b9cfb65-q6g9p\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") " pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:03.019527 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:03.019490    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-tmp-dir\") pod \"test-model-7b8c26-kserve-585b9cfb65-q6g9p\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") " pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:03.019682 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:03.019537    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-home\") pod \"test-model-7b8c26-kserve-585b9cfb65-q6g9p\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") " pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:03.019682 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:03.019611    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-model-cache\") pod \"test-model-7b8c26-kserve-585b9cfb65-q6g9p\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") " pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:03.021482 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:03.021464    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-dshm\") pod \"test-model-7b8c26-kserve-585b9cfb65-q6g9p\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") " pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:03.021682 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:03.021665    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b162c536-4383-4027-a940-9d4c29ed6bca-tls-certs\") pod \"test-model-7b8c26-kserve-585b9cfb65-q6g9p\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") " pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:03.027350 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:03.027333    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z5z8\" (UniqueName: \"kubernetes.io/projected/b162c536-4383-4027-a940-9d4c29ed6bca-kube-api-access-8z5z8\") pod \"test-model-7b8c26-kserve-585b9cfb65-q6g9p\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") " pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:03.112820 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:03.112731    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:03.271582 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:03.271554    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"]
Jul 13 20:01:03.273454 ip-10-0-143-91 kubenswrapper[2573]: W0713 20:01:03.273424    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb162c536_4383_4027_a940_9d4c29ed6bca.slice/crio-b1b7a0e89e8a20363aed45f31b4fdd770f21f27a202a3a586cab1c5a2880674c WatchSource:0}: Error finding container b1b7a0e89e8a20363aed45f31b4fdd770f21f27a202a3a586cab1c5a2880674c: Status 404 returned error can't find the container with id b1b7a0e89e8a20363aed45f31b4fdd770f21f27a202a3a586cab1c5a2880674c
Jul 13 20:01:04.149026 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:04.148991    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p" event={"ID":"b162c536-4383-4027-a940-9d4c29ed6bca","Type":"ContainerStarted","Data":"f89432a4a7bb52a402e76736030231784dbf155fb170d25e31778b2362672185"}
Jul 13 20:01:04.149026 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:04.149033    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p" event={"ID":"b162c536-4383-4027-a940-9d4c29ed6bca","Type":"ContainerStarted","Data":"b1b7a0e89e8a20363aed45f31b4fdd770f21f27a202a3a586cab1c5a2880674c"}
Jul 13 20:01:04.167669 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:04.167601    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p" podStartSLOduration=2.167585379 podStartE2EDuration="2.167585379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:01:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:01:04.166178102 +0000 UTC m=+2361.680054707" watchObservedRunningTime="2026-07-13 20:01:04.167585379 +0000 UTC m=+2361.681462018"
Jul 13 20:01:13.113077 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:13.113035    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:13.113589 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:13.113132    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:13.125665 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:13.125612    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:13.211903 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:13.211871    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:31.617772 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:31.617733    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"]
Jul 13 20:01:32.719161 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:32.719125    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"]
Jul 13 20:01:32.843576 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:32.843544    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dffe11-p8zlh"]
Jul 13 20:01:32.847133 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:32.847117    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dffe11-p8zlh"
Jul 13 20:01:32.849536 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:32.849514    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q5n9g\""
Jul 13 20:01:32.849672 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:32.849512    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 20:01:32.854199 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:32.854176    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dffe11-p8zlh"]
Jul 13 20:01:32.979891 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:32.979802    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea1d6bff-86fb-48fb-a821-61ac3fbde62f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-dffe11-p8zlh\" (UID: \"ea1d6bff-86fb-48fb-a821-61ac3fbde62f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dffe11-p8zlh"
Jul 13 20:01:33.080756 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:33.080727    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea1d6bff-86fb-48fb-a821-61ac3fbde62f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-dffe11-p8zlh\" (UID: \"ea1d6bff-86fb-48fb-a821-61ac3fbde62f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dffe11-p8zlh"
Jul 13 20:01:33.081365 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:33.081347    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea1d6bff-86fb-48fb-a821-61ac3fbde62f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-dffe11-p8zlh\" (UID: \"ea1d6bff-86fb-48fb-a821-61ac3fbde62f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dffe11-p8zlh"
Jul 13 20:01:33.158788 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:33.158750    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dffe11-p8zlh"
Jul 13 20:01:33.318070 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:33.317895    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dffe11-p8zlh"]
Jul 13 20:01:33.320971 ip-10-0-143-91 kubenswrapper[2573]: W0713 20:01:33.320943    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea1d6bff_86fb_48fb_a821_61ac3fbde62f.slice/crio-7b9a0e45410f62e7a065e44ec8a5c473c331a08677f40105b38cd969e3179505 WatchSource:0}: Error finding container 7b9a0e45410f62e7a065e44ec8a5c473c331a08677f40105b38cd969e3179505: Status 404 returned error can't find the container with id 7b9a0e45410f62e7a065e44ec8a5c473c331a08677f40105b38cd969e3179505
Jul 13 20:01:34.304384 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:34.304305    2573 generic.go:356] "Generic (PLEG): container finished" podID="ea1d6bff-86fb-48fb-a821-61ac3fbde62f" containerID="cee0c17a93c2a60ef77d76dae1b34ed5a3f18d7e6c119089bd384a372acfaccb" exitCode=0
Jul 13 20:01:34.304775 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:34.304390    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dffe11-p8zlh" event={"ID":"ea1d6bff-86fb-48fb-a821-61ac3fbde62f","Type":"ContainerDied","Data":"cee0c17a93c2a60ef77d76dae1b34ed5a3f18d7e6c119089bd384a372acfaccb"}
Jul 13 20:01:34.304775 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:34.304442    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dffe11-p8zlh" event={"ID":"ea1d6bff-86fb-48fb-a821-61ac3fbde62f","Type":"ContainerStarted","Data":"7b9a0e45410f62e7a065e44ec8a5c473c331a08677f40105b38cd969e3179505"}
Jul 13 20:01:35.441678 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:35.441628    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dffe11-p8zlh"
Jul 13 20:01:35.505421 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:35.505381    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea1d6bff-86fb-48fb-a821-61ac3fbde62f-maas-api-service-ca\") pod \"ea1d6bff-86fb-48fb-a821-61ac3fbde62f\" (UID: \"ea1d6bff-86fb-48fb-a821-61ac3fbde62f\") "
Jul 13 20:01:35.505797 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:35.505776    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea1d6bff-86fb-48fb-a821-61ac3fbde62f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ea1d6bff-86fb-48fb-a821-61ac3fbde62f" (UID: "ea1d6bff-86fb-48fb-a821-61ac3fbde62f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 20:01:35.606934 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:35.606839    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea1d6bff-86fb-48fb-a821-61ac3fbde62f-maas-api-service-ca\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:01:36.317232 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:36.317201    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dffe11-p8zlh"
Jul 13 20:01:36.317407 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:36.317229    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dffe11-p8zlh" event={"ID":"ea1d6bff-86fb-48fb-a821-61ac3fbde62f","Type":"ContainerDied","Data":"7b9a0e45410f62e7a065e44ec8a5c473c331a08677f40105b38cd969e3179505"}
Jul 13 20:01:36.317407 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:36.317278    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b9a0e45410f62e7a065e44ec8a5c473c331a08677f40105b38cd969e3179505"
Jul 13 20:01:43.073249 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:43.073222    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 20:01:43.075555 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:43.075523    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 20:01:43.076449 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:43.076424    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 20:01:43.079016 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:43.078996    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 20:01:46.638280 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:46.638230    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d" podUID="e610ab67-33e3-4dd0-aa38-fc5ac4835cef" containerName="main" containerID="cri-o://3adf43a4b9df660e4e2cc185da2b94d6a2a11a0bacb74fed45175a1ab9a0d9ab" gracePeriod=45
Jul 13 20:01:46.891487 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:46.891426    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:01:47.012803 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.012765    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-tmp-dir\") pod \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") "
Jul 13 20:01:47.012991 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.012869    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxq76\" (UniqueName: \"kubernetes.io/projected/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-kube-api-access-fxq76\") pod \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") "
Jul 13 20:01:47.012991 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.012895    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-tls-certs\") pod \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") "
Jul 13 20:01:47.012991 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.012911    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-home\") pod \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") "
Jul 13 20:01:47.012991 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.012961    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-dshm\") pod \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") "
Jul 13 20:01:47.013217 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.013028    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-model-cache\") pod \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\" (UID: \"e610ab67-33e3-4dd0-aa38-fc5ac4835cef\") "
Jul 13 20:01:47.013217 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.013069    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e610ab67-33e3-4dd0-aa38-fc5ac4835cef" (UID: "e610ab67-33e3-4dd0-aa38-fc5ac4835cef"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:01:47.013329 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.013238    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-home" (OuterVolumeSpecName: "home") pod "e610ab67-33e3-4dd0-aa38-fc5ac4835cef" (UID: "e610ab67-33e3-4dd0-aa38-fc5ac4835cef"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:01:47.013329 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.013257    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-model-cache" (OuterVolumeSpecName: "model-cache") pod "e610ab67-33e3-4dd0-aa38-fc5ac4835cef" (UID: "e610ab67-33e3-4dd0-aa38-fc5ac4835cef"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:01:47.013446 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.013333    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-home\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:01:47.013446 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.013350    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-model-cache\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:01:47.013446 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.013372    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-tmp-dir\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:01:47.015238 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.015216    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-dshm" (OuterVolumeSpecName: "dshm") pod "e610ab67-33e3-4dd0-aa38-fc5ac4835cef" (UID: "e610ab67-33e3-4dd0-aa38-fc5ac4835cef"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:01:47.015335 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.015294    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-kube-api-access-fxq76" (OuterVolumeSpecName: "kube-api-access-fxq76") pod "e610ab67-33e3-4dd0-aa38-fc5ac4835cef" (UID: "e610ab67-33e3-4dd0-aa38-fc5ac4835cef"). InnerVolumeSpecName "kube-api-access-fxq76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:01:47.015335 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.015312    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e610ab67-33e3-4dd0-aa38-fc5ac4835cef" (UID: "e610ab67-33e3-4dd0-aa38-fc5ac4835cef"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 20:01:47.114542 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.114507    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxq76\" (UniqueName: \"kubernetes.io/projected/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-kube-api-access-fxq76\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:01:47.114542 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.114536    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-tls-certs\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:01:47.114542 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.114547    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e610ab67-33e3-4dd0-aa38-fc5ac4835cef-dshm\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:01:47.374048 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.374014    2573 generic.go:356] "Generic (PLEG): container finished" podID="e610ab67-33e3-4dd0-aa38-fc5ac4835cef" containerID="3adf43a4b9df660e4e2cc185da2b94d6a2a11a0bacb74fed45175a1ab9a0d9ab" exitCode=0
Jul 13 20:01:47.374250 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.374057    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"
Jul 13 20:01:47.374250 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.374074    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d" event={"ID":"e610ab67-33e3-4dd0-aa38-fc5ac4835cef","Type":"ContainerDied","Data":"3adf43a4b9df660e4e2cc185da2b94d6a2a11a0bacb74fed45175a1ab9a0d9ab"}
Jul 13 20:01:47.374250 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.374112    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d" event={"ID":"e610ab67-33e3-4dd0-aa38-fc5ac4835cef","Type":"ContainerDied","Data":"9b60f3269c88a8f95ce915ecd29862241f0aaee314db903d7286adee0a1a9c82"}
Jul 13 20:01:47.374250 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.374134    2573 scope.go:117] "RemoveContainer" containerID="3adf43a4b9df660e4e2cc185da2b94d6a2a11a0bacb74fed45175a1ab9a0d9ab"
Jul 13 20:01:47.384033 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.384006    2573 scope.go:117] "RemoveContainer" containerID="3adf43a4b9df660e4e2cc185da2b94d6a2a11a0bacb74fed45175a1ab9a0d9ab"
Jul 13 20:01:47.384286 ip-10-0-143-91 kubenswrapper[2573]: E0713 20:01:47.384264    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3adf43a4b9df660e4e2cc185da2b94d6a2a11a0bacb74fed45175a1ab9a0d9ab\": container with ID starting with 3adf43a4b9df660e4e2cc185da2b94d6a2a11a0bacb74fed45175a1ab9a0d9ab not found: ID does not exist" containerID="3adf43a4b9df660e4e2cc185da2b94d6a2a11a0bacb74fed45175a1ab9a0d9ab"
Jul 13 20:01:47.384362 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.384295    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3adf43a4b9df660e4e2cc185da2b94d6a2a11a0bacb74fed45175a1ab9a0d9ab"} err="failed to get container status \"3adf43a4b9df660e4e2cc185da2b94d6a2a11a0bacb74fed45175a1ab9a0d9ab\": rpc error: code = NotFound desc = could not find container \"3adf43a4b9df660e4e2cc185da2b94d6a2a11a0bacb74fed45175a1ab9a0d9ab\": container with ID starting with 3adf43a4b9df660e4e2cc185da2b94d6a2a11a0bacb74fed45175a1ab9a0d9ab not found: ID does not exist"
Jul 13 20:01:47.391263 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.391228    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"]
Jul 13 20:01:47.395159 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.395134    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-dffe11/test-model-dffe11-kserve-6b9968d55f-wnd6d"]
Jul 13 20:01:47.740160 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.740118    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p" podUID="b162c536-4383-4027-a940-9d4c29ed6bca" containerName="main" containerID="cri-o://f89432a4a7bb52a402e76736030231784dbf155fb170d25e31778b2362672185" gracePeriod=45
Jul 13 20:01:47.996026 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:47.995966    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:48.121941 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.121900    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b162c536-4383-4027-a940-9d4c29ed6bca-tls-certs\") pod \"b162c536-4383-4027-a940-9d4c29ed6bca\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") "
Jul 13 20:01:48.122139 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.121975    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-dshm\") pod \"b162c536-4383-4027-a940-9d4c29ed6bca\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") "
Jul 13 20:01:48.122139 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.122001    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-home\") pod \"b162c536-4383-4027-a940-9d4c29ed6bca\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") "
Jul 13 20:01:48.122139 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.122049    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-tmp-dir\") pod \"b162c536-4383-4027-a940-9d4c29ed6bca\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") "
Jul 13 20:01:48.122139 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.122071    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8z5z8\" (UniqueName: \"kubernetes.io/projected/b162c536-4383-4027-a940-9d4c29ed6bca-kube-api-access-8z5z8\") pod \"b162c536-4383-4027-a940-9d4c29ed6bca\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") "
Jul 13 20:01:48.122139 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.122116    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-model-cache\") pod \"b162c536-4383-4027-a940-9d4c29ed6bca\" (UID: \"b162c536-4383-4027-a940-9d4c29ed6bca\") "
Jul 13 20:01:48.122490 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.122438    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-home" (OuterVolumeSpecName: "home") pod "b162c536-4383-4027-a940-9d4c29ed6bca" (UID: "b162c536-4383-4027-a940-9d4c29ed6bca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:01:48.122715 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.122684    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b162c536-4383-4027-a940-9d4c29ed6bca" (UID: "b162c536-4383-4027-a940-9d4c29ed6bca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:01:48.122833 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.122551    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-model-cache" (OuterVolumeSpecName: "model-cache") pod "b162c536-4383-4027-a940-9d4c29ed6bca" (UID: "b162c536-4383-4027-a940-9d4c29ed6bca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:01:48.124320 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.124295    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-dshm" (OuterVolumeSpecName: "dshm") pod "b162c536-4383-4027-a940-9d4c29ed6bca" (UID: "b162c536-4383-4027-a940-9d4c29ed6bca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:01:48.124430 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.124395    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b162c536-4383-4027-a940-9d4c29ed6bca-kube-api-access-8z5z8" (OuterVolumeSpecName: "kube-api-access-8z5z8") pod "b162c536-4383-4027-a940-9d4c29ed6bca" (UID: "b162c536-4383-4027-a940-9d4c29ed6bca"). InnerVolumeSpecName "kube-api-access-8z5z8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:01:48.124523 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.124489    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b162c536-4383-4027-a940-9d4c29ed6bca-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b162c536-4383-4027-a940-9d4c29ed6bca" (UID: "b162c536-4383-4027-a940-9d4c29ed6bca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 20:01:48.223008 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.222959    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-dshm\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:01:48.223008 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.223003    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-home\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:01:48.223008 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.223013    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-tmp-dir\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:01:48.223008 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.223025    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8z5z8\" (UniqueName: \"kubernetes.io/projected/b162c536-4383-4027-a940-9d4c29ed6bca-kube-api-access-8z5z8\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:01:48.223292 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.223036    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b162c536-4383-4027-a940-9d4c29ed6bca-model-cache\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:01:48.223292 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.223044    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b162c536-4383-4027-a940-9d4c29ed6bca-tls-certs\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:01:48.380241 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.380147    2573 generic.go:356] "Generic (PLEG): container finished" podID="b162c536-4383-4027-a940-9d4c29ed6bca" containerID="f89432a4a7bb52a402e76736030231784dbf155fb170d25e31778b2362672185" exitCode=0
Jul 13 20:01:48.380241 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.380201    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"
Jul 13 20:01:48.380487 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.380236    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p" event={"ID":"b162c536-4383-4027-a940-9d4c29ed6bca","Type":"ContainerDied","Data":"f89432a4a7bb52a402e76736030231784dbf155fb170d25e31778b2362672185"}
Jul 13 20:01:48.380487 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.380286    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p" event={"ID":"b162c536-4383-4027-a940-9d4c29ed6bca","Type":"ContainerDied","Data":"b1b7a0e89e8a20363aed45f31b4fdd770f21f27a202a3a586cab1c5a2880674c"}
Jul 13 20:01:48.380487 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.380307    2573 scope.go:117] "RemoveContainer" containerID="f89432a4a7bb52a402e76736030231784dbf155fb170d25e31778b2362672185"
Jul 13 20:01:48.396668 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.396625    2573 scope.go:117] "RemoveContainer" containerID="f89432a4a7bb52a402e76736030231784dbf155fb170d25e31778b2362672185"
Jul 13 20:01:48.396964 ip-10-0-143-91 kubenswrapper[2573]: E0713 20:01:48.396945    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f89432a4a7bb52a402e76736030231784dbf155fb170d25e31778b2362672185\": container with ID starting with f89432a4a7bb52a402e76736030231784dbf155fb170d25e31778b2362672185 not found: ID does not exist" containerID="f89432a4a7bb52a402e76736030231784dbf155fb170d25e31778b2362672185"
Jul 13 20:01:48.397012 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.396974    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f89432a4a7bb52a402e76736030231784dbf155fb170d25e31778b2362672185"} err="failed to get container status \"f89432a4a7bb52a402e76736030231784dbf155fb170d25e31778b2362672185\": rpc error: code = NotFound desc = could not find container \"f89432a4a7bb52a402e76736030231784dbf155fb170d25e31778b2362672185\": container with ID starting with f89432a4a7bb52a402e76736030231784dbf155fb170d25e31778b2362672185 not found: ID does not exist"
Jul 13 20:01:48.405386 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.405361    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"]
Jul 13 20:01:48.411419 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:48.411395    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-7b8c26/test-model-7b8c26-kserve-585b9cfb65-q6g9p"]
Jul 13 20:01:49.048650 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:49.048612    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b162c536-4383-4027-a940-9d4c29ed6bca" path="/var/lib/kubelet/pods/b162c536-4383-4027-a940-9d4c29ed6bca/volumes"
Jul 13 20:01:49.049039 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:49.049024    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e610ab67-33e3-4dd0-aa38-fc5ac4835cef" path="/var/lib/kubelet/pods/e610ab67-33e3-4dd0-aa38-fc5ac4835cef/volumes"
Jul 13 20:01:58.968578 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:58.968542    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7b8c26-zs7pl"]
Jul 13 20:01:58.968984 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:58.968887    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea1d6bff-86fb-48fb-a821-61ac3fbde62f" containerName="revoke-keys"
Jul 13 20:01:58.968984 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:58.968899    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea1d6bff-86fb-48fb-a821-61ac3fbde62f" containerName="revoke-keys"
Jul 13 20:01:58.968984 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:58.968912    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e610ab67-33e3-4dd0-aa38-fc5ac4835cef" containerName="main"
Jul 13 20:01:58.968984 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:58.968919    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e610ab67-33e3-4dd0-aa38-fc5ac4835cef" containerName="main"
Jul 13 20:01:58.968984 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:58.968932    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b162c536-4383-4027-a940-9d4c29ed6bca" containerName="main"
Jul 13 20:01:58.968984 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:58.968938    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b162c536-4383-4027-a940-9d4c29ed6bca" containerName="main"
Jul 13 20:01:58.969175 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:58.968995    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea1d6bff-86fb-48fb-a821-61ac3fbde62f" containerName="revoke-keys"
Jul 13 20:01:58.969175 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:58.969003    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e610ab67-33e3-4dd0-aa38-fc5ac4835cef" containerName="main"
Jul 13 20:01:58.969175 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:58.969013    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b162c536-4383-4027-a940-9d4c29ed6bca" containerName="main"
Jul 13 20:01:58.973444 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:58.973428    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7b8c26-zs7pl"
Jul 13 20:01:58.976187 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:58.976128    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 20:01:58.976322 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:58.976200    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q5n9g\""
Jul 13 20:01:58.979891 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:58.979847    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7b8c26-zs7pl"]
Jul 13 20:01:59.020704 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:59.020630    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27538973-2037-4ce5-bd50-4feb6c35f108-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7b8c26-zs7pl\" (UID: \"27538973-2037-4ce5-bd50-4feb6c35f108\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7b8c26-zs7pl"
Jul 13 20:01:59.121131 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:59.121096    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27538973-2037-4ce5-bd50-4feb6c35f108-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7b8c26-zs7pl\" (UID: \"27538973-2037-4ce5-bd50-4feb6c35f108\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7b8c26-zs7pl"
Jul 13 20:01:59.121831 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:59.121809    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27538973-2037-4ce5-bd50-4feb6c35f108-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7b8c26-zs7pl\" (UID: \"27538973-2037-4ce5-bd50-4feb6c35f108\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7b8c26-zs7pl"
Jul 13 20:01:59.285133 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:59.285045    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7b8c26-zs7pl"
Jul 13 20:01:59.418531 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:59.418505    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7b8c26-zs7pl"]
Jul 13 20:01:59.420699 ip-10-0-143-91 kubenswrapper[2573]: W0713 20:01:59.420668    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27538973_2037_4ce5_bd50_4feb6c35f108.slice/crio-38b831875c5d831395c2414cf52fb70c3ef445c3d3cdbb7487970c66f33e649f WatchSource:0}: Error finding container 38b831875c5d831395c2414cf52fb70c3ef445c3d3cdbb7487970c66f33e649f: Status 404 returned error can't find the container with id 38b831875c5d831395c2414cf52fb70c3ef445c3d3cdbb7487970c66f33e649f
Jul 13 20:01:59.439560 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:01:59.439530    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7b8c26-zs7pl" event={"ID":"27538973-2037-4ce5-bd50-4feb6c35f108","Type":"ContainerStarted","Data":"38b831875c5d831395c2414cf52fb70c3ef445c3d3cdbb7487970c66f33e649f"}
Jul 13 20:02:00.446060 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:00.446026    2573 generic.go:356] "Generic (PLEG): container finished" podID="27538973-2037-4ce5-bd50-4feb6c35f108" containerID="4867c8141e7426b3b3b0a50740001d79c920465799185aa2edc4348f05fcbd15" exitCode=0
Jul 13 20:02:00.446487 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:00.446109    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7b8c26-zs7pl" event={"ID":"27538973-2037-4ce5-bd50-4feb6c35f108","Type":"ContainerDied","Data":"4867c8141e7426b3b3b0a50740001d79c920465799185aa2edc4348f05fcbd15"}
Jul 13 20:02:01.580821 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:01.580794    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7b8c26-zs7pl"
Jul 13 20:02:01.639063 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:01.639009    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27538973-2037-4ce5-bd50-4feb6c35f108-maas-api-service-ca\") pod \"27538973-2037-4ce5-bd50-4feb6c35f108\" (UID: \"27538973-2037-4ce5-bd50-4feb6c35f108\") "
Jul 13 20:02:01.639459 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:01.639433    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27538973-2037-4ce5-bd50-4feb6c35f108-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "27538973-2037-4ce5-bd50-4feb6c35f108" (UID: "27538973-2037-4ce5-bd50-4feb6c35f108"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 20:02:01.740228 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:01.740141    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27538973-2037-4ce5-bd50-4feb6c35f108-maas-api-service-ca\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:02:02.456413 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:02.456385    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7b8c26-zs7pl"
Jul 13 20:02:02.456602 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:02.456382    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7b8c26-zs7pl" event={"ID":"27538973-2037-4ce5-bd50-4feb6c35f108","Type":"ContainerDied","Data":"38b831875c5d831395c2414cf52fb70c3ef445c3d3cdbb7487970c66f33e649f"}
Jul 13 20:02:02.456602 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:02.456514    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="38b831875c5d831395c2414cf52fb70c3ef445c3d3cdbb7487970c66f33e649f"
Jul 13 20:02:48.892109 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:48.892071    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519ff3-d9ddk"]
Jul 13 20:02:48.892577 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:48.892455    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27538973-2037-4ce5-bd50-4feb6c35f108" containerName="revoke-keys"
Jul 13 20:02:48.892577 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:48.892467    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="27538973-2037-4ce5-bd50-4feb6c35f108" containerName="revoke-keys"
Jul 13 20:02:48.892577 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:48.892542    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="27538973-2037-4ce5-bd50-4feb6c35f108" containerName="revoke-keys"
Jul 13 20:02:48.895687 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:48.895670    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519ff3-d9ddk"
Jul 13 20:02:48.898205 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:48.898179    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q5n9g\""
Jul 13 20:02:48.898356 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:48.898186    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 20:02:48.903469 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:48.903038    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519ff3-d9ddk"]
Jul 13 20:02:48.960684 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:48.960624    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d3ba106-dc46-4d92-aa60-52803d063c40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-519ff3-d9ddk\" (UID: \"1d3ba106-dc46-4d92-aa60-52803d063c40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519ff3-d9ddk"
Jul 13 20:02:49.061236 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:49.061208    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d3ba106-dc46-4d92-aa60-52803d063c40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-519ff3-d9ddk\" (UID: \"1d3ba106-dc46-4d92-aa60-52803d063c40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519ff3-d9ddk"
Jul 13 20:02:49.061870 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:49.061850    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d3ba106-dc46-4d92-aa60-52803d063c40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-519ff3-d9ddk\" (UID: \"1d3ba106-dc46-4d92-aa60-52803d063c40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519ff3-d9ddk"
Jul 13 20:02:49.207530 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:49.207496    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519ff3-d9ddk"
Jul 13 20:02:49.345357 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:49.345334    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519ff3-d9ddk"]
Jul 13 20:02:49.347009 ip-10-0-143-91 kubenswrapper[2573]: W0713 20:02:49.346978    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d3ba106_dc46_4d92_aa60_52803d063c40.slice/crio-43d6fbb3e6a29489a04526492fe6e23b71a7838d00c87eaf1e778a57e1c82887 WatchSource:0}: Error finding container 43d6fbb3e6a29489a04526492fe6e23b71a7838d00c87eaf1e778a57e1c82887: Status 404 returned error can't find the container with id 43d6fbb3e6a29489a04526492fe6e23b71a7838d00c87eaf1e778a57e1c82887
Jul 13 20:02:49.688439 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:49.688406    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519ff3-d9ddk" event={"ID":"1d3ba106-dc46-4d92-aa60-52803d063c40","Type":"ContainerStarted","Data":"fb7e2756e9b4d029d3d12634cb07751774f2fa71a992cfd7c7e1e5028484c37e"}
Jul 13 20:02:49.688663 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:49.688467    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519ff3-d9ddk" event={"ID":"1d3ba106-dc46-4d92-aa60-52803d063c40","Type":"ContainerStarted","Data":"43d6fbb3e6a29489a04526492fe6e23b71a7838d00c87eaf1e778a57e1c82887"}
Jul 13 20:02:49.706706 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:49.706626    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519ff3-d9ddk" podStartSLOduration=1.706604638 podStartE2EDuration="1.706604638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:02:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:02:49.703334673 +0000 UTC m=+2467.217211278" watchObservedRunningTime="2026-07-13 20:02:49.706604638 +0000 UTC m=+2467.220481245"
Jul 13 20:02:50.694899 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:50.694865    2573 generic.go:356] "Generic (PLEG): container finished" podID="1d3ba106-dc46-4d92-aa60-52803d063c40" containerID="fb7e2756e9b4d029d3d12634cb07751774f2fa71a992cfd7c7e1e5028484c37e" exitCode=0
Jul 13 20:02:50.695315 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:50.694950    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519ff3-d9ddk" event={"ID":"1d3ba106-dc46-4d92-aa60-52803d063c40","Type":"ContainerDied","Data":"fb7e2756e9b4d029d3d12634cb07751774f2fa71a992cfd7c7e1e5028484c37e"}
Jul 13 20:02:51.834654 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:51.834616    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519ff3-d9ddk"
Jul 13 20:02:51.887422 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:51.887389    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d3ba106-dc46-4d92-aa60-52803d063c40-maas-api-service-ca\") pod \"1d3ba106-dc46-4d92-aa60-52803d063c40\" (UID: \"1d3ba106-dc46-4d92-aa60-52803d063c40\") "
Jul 13 20:02:51.887773 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:51.887752    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d3ba106-dc46-4d92-aa60-52803d063c40-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1d3ba106-dc46-4d92-aa60-52803d063c40" (UID: "1d3ba106-dc46-4d92-aa60-52803d063c40"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 20:02:51.989063 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:51.988971    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d3ba106-dc46-4d92-aa60-52803d063c40-maas-api-service-ca\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:02:52.706025 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:52.705979    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519ff3-d9ddk" event={"ID":"1d3ba106-dc46-4d92-aa60-52803d063c40","Type":"ContainerDied","Data":"43d6fbb3e6a29489a04526492fe6e23b71a7838d00c87eaf1e778a57e1c82887"}
Jul 13 20:02:52.706025 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:52.706015    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519ff3-d9ddk"
Jul 13 20:02:52.706250 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:02:52.706039    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="43d6fbb3e6a29489a04526492fe6e23b71a7838d00c87eaf1e778a57e1c82887"
Jul 13 20:03:15.047326 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:15.047293    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f221d9-mpzlw"]
Jul 13 20:03:15.047730 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:15.047605    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d3ba106-dc46-4d92-aa60-52803d063c40" containerName="revoke-keys"
Jul 13 20:03:15.047730 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:15.047615    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d3ba106-dc46-4d92-aa60-52803d063c40" containerName="revoke-keys"
Jul 13 20:03:15.047730 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:15.047704    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d3ba106-dc46-4d92-aa60-52803d063c40" containerName="revoke-keys"
Jul 13 20:03:15.051020 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:15.050993    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f221d9-mpzlw"
Jul 13 20:03:15.052244 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:15.052214    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f221d9-mpzlw"]
Jul 13 20:03:15.053975 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:15.053955    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 20:03:15.055006 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:15.054938    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q5n9g\""
Jul 13 20:03:15.201008 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:15.200962    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec52eb22-00ab-435f-a826-87327f559716-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f221d9-mpzlw\" (UID: \"ec52eb22-00ab-435f-a826-87327f559716\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f221d9-mpzlw"
Jul 13 20:03:15.302175 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:15.302059    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec52eb22-00ab-435f-a826-87327f559716-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f221d9-mpzlw\" (UID: \"ec52eb22-00ab-435f-a826-87327f559716\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f221d9-mpzlw"
Jul 13 20:03:15.302760 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:15.302738    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec52eb22-00ab-435f-a826-87327f559716-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f221d9-mpzlw\" (UID: \"ec52eb22-00ab-435f-a826-87327f559716\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f221d9-mpzlw"
Jul 13 20:03:15.362424 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:15.362382    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f221d9-mpzlw"
Jul 13 20:03:15.496317 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:15.496292    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f221d9-mpzlw"]
Jul 13 20:03:15.498125 ip-10-0-143-91 kubenswrapper[2573]: W0713 20:03:15.498096    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec52eb22_00ab_435f_a826_87327f559716.slice/crio-857c3ddc72a8f6d91108df46968457af4522f911d784cd6bab68be13c0090fd0 WatchSource:0}: Error finding container 857c3ddc72a8f6d91108df46968457af4522f911d784cd6bab68be13c0090fd0: Status 404 returned error can't find the container with id 857c3ddc72a8f6d91108df46968457af4522f911d784cd6bab68be13c0090fd0
Jul 13 20:03:15.829125 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:15.829084    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f221d9-mpzlw" event={"ID":"ec52eb22-00ab-435f-a826-87327f559716","Type":"ContainerStarted","Data":"253c510d870971f823aafc098a10db9049d8994b3e4fbcd2d6db3ae01c7f7048"}
Jul 13 20:03:15.829298 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:15.829151    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f221d9-mpzlw" event={"ID":"ec52eb22-00ab-435f-a826-87327f559716","Type":"ContainerStarted","Data":"857c3ddc72a8f6d91108df46968457af4522f911d784cd6bab68be13c0090fd0"}
Jul 13 20:03:15.849157 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:15.849107    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f221d9-mpzlw" podStartSLOduration=0.849089473 podStartE2EDuration="849.089473ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:03:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:03:15.847922868 +0000 UTC m=+2493.361799495" watchObservedRunningTime="2026-07-13 20:03:15.849089473 +0000 UTC m=+2493.362966079"
Jul 13 20:03:16.835847 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:16.835808    2573 generic.go:356] "Generic (PLEG): container finished" podID="ec52eb22-00ab-435f-a826-87327f559716" containerID="253c510d870971f823aafc098a10db9049d8994b3e4fbcd2d6db3ae01c7f7048" exitCode=0
Jul 13 20:03:16.836243 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:16.835877    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f221d9-mpzlw" event={"ID":"ec52eb22-00ab-435f-a826-87327f559716","Type":"ContainerDied","Data":"253c510d870971f823aafc098a10db9049d8994b3e4fbcd2d6db3ae01c7f7048"}
Jul 13 20:03:17.971984 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:17.971959    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f221d9-mpzlw"
Jul 13 20:03:18.129926 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:18.129824    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec52eb22-00ab-435f-a826-87327f559716-maas-api-service-ca\") pod \"ec52eb22-00ab-435f-a826-87327f559716\" (UID: \"ec52eb22-00ab-435f-a826-87327f559716\") "
Jul 13 20:03:18.130189 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:18.130167    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec52eb22-00ab-435f-a826-87327f559716-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ec52eb22-00ab-435f-a826-87327f559716" (UID: "ec52eb22-00ab-435f-a826-87327f559716"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 20:03:18.231243 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:18.231201    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec52eb22-00ab-435f-a826-87327f559716-maas-api-service-ca\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:03:18.847122 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:18.847086    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f221d9-mpzlw" event={"ID":"ec52eb22-00ab-435f-a826-87327f559716","Type":"ContainerDied","Data":"857c3ddc72a8f6d91108df46968457af4522f911d784cd6bab68be13c0090fd0"}
Jul 13 20:03:18.847304 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:18.847161    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="857c3ddc72a8f6d91108df46968457af4522f911d784cd6bab68be13c0090fd0"
Jul 13 20:03:18.847304 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:03:18.847103    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f221d9-mpzlw"
Jul 13 20:04:06.808220 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:06.808133    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"]
Jul 13 20:04:06.808741 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:06.808510    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec52eb22-00ab-435f-a826-87327f559716" containerName="revoke-keys"
Jul 13 20:04:06.808741 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:06.808521    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec52eb22-00ab-435f-a826-87327f559716" containerName="revoke-keys"
Jul 13 20:04:06.808741 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:06.808574    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec52eb22-00ab-435f-a826-87327f559716" containerName="revoke-keys"
Jul 13 20:04:06.811750 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:06.811733    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:06.815289 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:06.815262    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-452240\"/\"default-dockercfg-jczhj\""
Jul 13 20:04:06.815289 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:06.815262    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-452240\"/\"kube-root-ca.crt\""
Jul 13 20:04:06.815492 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:06.815304    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-452240\"/\"openshift-service-ca.crt\""
Jul 13 20:04:06.815492 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:06.815288    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-452240\"/\"auth-test-model-452240-kserve-self-signed-certs\""
Jul 13 20:04:06.819867 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:06.819845    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"]
Jul 13 20:04:06.985180 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:06.985138    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-home\") pod \"auth-test-model-452240-kserve-68c68bb447-xmnw7\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") " pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:06.985180 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:06.985182    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd56f990-cc2c-465c-b17f-7900fc71ad4e-tls-certs\") pod \"auth-test-model-452240-kserve-68c68bb447-xmnw7\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") " pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:06.985441 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:06.985237    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-model-cache\") pod \"auth-test-model-452240-kserve-68c68bb447-xmnw7\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") " pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:06.985441 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:06.985268    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxhjx\" (UniqueName: \"kubernetes.io/projected/cd56f990-cc2c-465c-b17f-7900fc71ad4e-kube-api-access-lxhjx\") pod \"auth-test-model-452240-kserve-68c68bb447-xmnw7\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") " pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:06.985441 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:06.985361    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-tmp-dir\") pod \"auth-test-model-452240-kserve-68c68bb447-xmnw7\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") " pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:06.985441 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:06.985433    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-dshm\") pod \"auth-test-model-452240-kserve-68c68bb447-xmnw7\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") " pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:07.086158 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:07.086065    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-dshm\") pod \"auth-test-model-452240-kserve-68c68bb447-xmnw7\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") " pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:07.086158 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:07.086121    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-home\") pod \"auth-test-model-452240-kserve-68c68bb447-xmnw7\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") " pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:07.086158 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:07.086157    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd56f990-cc2c-465c-b17f-7900fc71ad4e-tls-certs\") pod \"auth-test-model-452240-kserve-68c68bb447-xmnw7\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") " pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:07.086447 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:07.086193    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-model-cache\") pod \"auth-test-model-452240-kserve-68c68bb447-xmnw7\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") " pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:07.086447 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:07.086240    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxhjx\" (UniqueName: \"kubernetes.io/projected/cd56f990-cc2c-465c-b17f-7900fc71ad4e-kube-api-access-lxhjx\") pod \"auth-test-model-452240-kserve-68c68bb447-xmnw7\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") " pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:07.086447 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:07.086279    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-tmp-dir\") pod \"auth-test-model-452240-kserve-68c68bb447-xmnw7\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") " pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:07.086662 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:07.086623    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-home\") pod \"auth-test-model-452240-kserve-68c68bb447-xmnw7\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") " pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:07.086744 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:07.086682    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-model-cache\") pod \"auth-test-model-452240-kserve-68c68bb447-xmnw7\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") " pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:07.086744 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:07.086721    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-tmp-dir\") pod \"auth-test-model-452240-kserve-68c68bb447-xmnw7\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") " pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:07.088648 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:07.088603    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-dshm\") pod \"auth-test-model-452240-kserve-68c68bb447-xmnw7\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") " pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:07.088802 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:07.088784    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd56f990-cc2c-465c-b17f-7900fc71ad4e-tls-certs\") pod \"auth-test-model-452240-kserve-68c68bb447-xmnw7\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") " pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:07.094148 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:07.094123    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxhjx\" (UniqueName: \"kubernetes.io/projected/cd56f990-cc2c-465c-b17f-7900fc71ad4e-kube-api-access-lxhjx\") pod \"auth-test-model-452240-kserve-68c68bb447-xmnw7\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") " pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:07.121345 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:07.121318    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:07.255515 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:07.255487    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"]
Jul 13 20:04:07.257471 ip-10-0-143-91 kubenswrapper[2573]: W0713 20:04:07.257429    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd56f990_cc2c_465c_b17f_7900fc71ad4e.slice/crio-9981f2705503cc2b6247691c772143c331fbaa0425f8118cc67295b24d1d07ed WatchSource:0}: Error finding container 9981f2705503cc2b6247691c772143c331fbaa0425f8118cc67295b24d1d07ed: Status 404 returned error can't find the container with id 9981f2705503cc2b6247691c772143c331fbaa0425f8118cc67295b24d1d07ed
Jul 13 20:04:08.092878 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:08.092842    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7" event={"ID":"cd56f990-cc2c-465c-b17f-7900fc71ad4e","Type":"ContainerStarted","Data":"bac456b9688f073a6e6eec648d6bb587083737784add035e27d960934705ad42"}
Jul 13 20:04:08.093292 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:08.092885    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7" event={"ID":"cd56f990-cc2c-465c-b17f-7900fc71ad4e","Type":"ContainerStarted","Data":"9981f2705503cc2b6247691c772143c331fbaa0425f8118cc67295b24d1d07ed"}
Jul 13 20:04:08.111333 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:08.111284    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7" podStartSLOduration=2.111268207 podStartE2EDuration="2.111268207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:04:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:04:08.11053557 +0000 UTC m=+2545.624412179" watchObservedRunningTime="2026-07-13 20:04:08.111268207 +0000 UTC m=+2545.625144813"
Jul 13 20:04:17.121769 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:17.121733    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:17.122267 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:17.121896    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:17.135258 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:17.135231    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:17.150234 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:17.150205    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:04:44.010961 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.010919    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"]
Jul 13 20:04:44.014846 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.014828    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:44.017520 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.017500    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-83eaac\"/\"openshift-service-ca.crt\""
Jul 13 20:04:44.017623 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.017539    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-83eaac\"/\"kube-root-ca.crt\""
Jul 13 20:04:44.018513 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.018492    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-83eaac\"/\"auth-test-model-83eaac-kserve-self-signed-certs\""
Jul 13 20:04:44.018614 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.018531    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-83eaac\"/\"default-dockercfg-v2cbt\""
Jul 13 20:04:44.023035 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.023015    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"]
Jul 13 20:04:44.103790 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.103755    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-dshm\") pod \"auth-test-model-83eaac-kserve-fd84cddcf-hb5gp\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") " pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:44.103977 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.103807    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-model-cache\") pod \"auth-test-model-83eaac-kserve-fd84cddcf-hb5gp\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") " pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:44.103977 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.103841    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-home\") pod \"auth-test-model-83eaac-kserve-fd84cddcf-hb5gp\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") " pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:44.103977 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.103871    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-tmp-dir\") pod \"auth-test-model-83eaac-kserve-fd84cddcf-hb5gp\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") " pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:44.103977 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.103886    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93b9a5e5-775e-45bc-bd9c-e058697fabec-tls-certs\") pod \"auth-test-model-83eaac-kserve-fd84cddcf-hb5gp\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") " pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:44.103977 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.103953    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjcfn\" (UniqueName: \"kubernetes.io/projected/93b9a5e5-775e-45bc-bd9c-e058697fabec-kube-api-access-mjcfn\") pod \"auth-test-model-83eaac-kserve-fd84cddcf-hb5gp\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") " pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:44.205160 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.205079    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjcfn\" (UniqueName: \"kubernetes.io/projected/93b9a5e5-775e-45bc-bd9c-e058697fabec-kube-api-access-mjcfn\") pod \"auth-test-model-83eaac-kserve-fd84cddcf-hb5gp\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") " pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:44.205367 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.205166    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-dshm\") pod \"auth-test-model-83eaac-kserve-fd84cddcf-hb5gp\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") " pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:44.205367 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.205216    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-model-cache\") pod \"auth-test-model-83eaac-kserve-fd84cddcf-hb5gp\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") " pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:44.205367 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.205251    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-home\") pod \"auth-test-model-83eaac-kserve-fd84cddcf-hb5gp\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") " pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:44.205367 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.205295    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-tmp-dir\") pod \"auth-test-model-83eaac-kserve-fd84cddcf-hb5gp\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") " pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:44.205367 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.205316    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93b9a5e5-775e-45bc-bd9c-e058697fabec-tls-certs\") pod \"auth-test-model-83eaac-kserve-fd84cddcf-hb5gp\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") " pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:44.205691 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.205666    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-home\") pod \"auth-test-model-83eaac-kserve-fd84cddcf-hb5gp\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") " pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:44.205766 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.205725    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-model-cache\") pod \"auth-test-model-83eaac-kserve-fd84cddcf-hb5gp\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") " pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:44.205766 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.205750    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-tmp-dir\") pod \"auth-test-model-83eaac-kserve-fd84cddcf-hb5gp\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") " pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:44.207464 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.207443    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-dshm\") pod \"auth-test-model-83eaac-kserve-fd84cddcf-hb5gp\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") " pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:44.207801 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.207784    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93b9a5e5-775e-45bc-bd9c-e058697fabec-tls-certs\") pod \"auth-test-model-83eaac-kserve-fd84cddcf-hb5gp\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") " pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:44.217728 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.217705    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjcfn\" (UniqueName: \"kubernetes.io/projected/93b9a5e5-775e-45bc-bd9c-e058697fabec-kube-api-access-mjcfn\") pod \"auth-test-model-83eaac-kserve-fd84cddcf-hb5gp\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") " pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:44.324473 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.324362    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:44.458825 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:44.458799    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"]
Jul 13 20:04:44.460391 ip-10-0-143-91 kubenswrapper[2573]: W0713 20:04:44.460355    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93b9a5e5_775e_45bc_bd9c_e058697fabec.slice/crio-22cd01d3902657b62131056da395516e46b4c1d563ed945a6f3d3e6b3d6d2ea7 WatchSource:0}: Error finding container 22cd01d3902657b62131056da395516e46b4c1d563ed945a6f3d3e6b3d6d2ea7: Status 404 returned error can't find the container with id 22cd01d3902657b62131056da395516e46b4c1d563ed945a6f3d3e6b3d6d2ea7
Jul 13 20:04:45.278070 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:45.278031    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp" event={"ID":"93b9a5e5-775e-45bc-bd9c-e058697fabec","Type":"ContainerStarted","Data":"1e7d1274d7e77073dcfa6bd22f13a5bf1fdfcdded599db8a047b2ce90cc0ebe3"}
Jul 13 20:04:45.278477 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:45.278080    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp" event={"ID":"93b9a5e5-775e-45bc-bd9c-e058697fabec","Type":"ContainerStarted","Data":"22cd01d3902657b62131056da395516e46b4c1d563ed945a6f3d3e6b3d6d2ea7"}
Jul 13 20:04:45.294013 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:45.293951    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp" podStartSLOduration=2.293929637 podStartE2EDuration="2.293929637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:04:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:04:45.293275578 +0000 UTC m=+2582.807152185" watchObservedRunningTime="2026-07-13 20:04:45.293929637 +0000 UTC m=+2582.807806244"
Jul 13 20:04:54.324589 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:54.324551    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:54.325062 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:54.324606    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:54.337378 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:54.337348    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:04:55.341255 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:04:55.341224    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:06:43.104825 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:06:43.104737    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 20:06:43.107575 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:06:43.107555    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 20:06:43.109411 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:06:43.109390    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 20:06:43.112177 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:06:43.112157    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 20:06:53.386569 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:06:53.386531    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"]
Jul 13 20:06:54.310548 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:06:54.310507    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"]
Jul 13 20:07:08.406821 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:08.406775    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7" podUID="cd56f990-cc2c-465c-b17f-7900fc71ad4e" containerName="main" containerID="cri-o://bac456b9688f073a6e6eec648d6bb587083737784add035e27d960934705ad42" gracePeriod=45
Jul 13 20:07:08.664203 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:08.664139    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:07:08.695937 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:08.695904    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxhjx\" (UniqueName: \"kubernetes.io/projected/cd56f990-cc2c-465c-b17f-7900fc71ad4e-kube-api-access-lxhjx\") pod \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") "
Jul 13 20:07:08.696114 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:08.695959    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-home\") pod \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") "
Jul 13 20:07:08.696114 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:08.695986    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-dshm\") pod \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") "
Jul 13 20:07:08.696114 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:08.696068    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-tmp-dir\") pod \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") "
Jul 13 20:07:08.696114 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:08.696096    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd56f990-cc2c-465c-b17f-7900fc71ad4e-tls-certs\") pod \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") "
Jul 13 20:07:08.696351 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:08.696123    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-model-cache\") pod \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\" (UID: \"cd56f990-cc2c-465c-b17f-7900fc71ad4e\") "
Jul 13 20:07:08.696351 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:08.696291    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-home" (OuterVolumeSpecName: "home") pod "cd56f990-cc2c-465c-b17f-7900fc71ad4e" (UID: "cd56f990-cc2c-465c-b17f-7900fc71ad4e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:07:08.696454 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:08.696385    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cd56f990-cc2c-465c-b17f-7900fc71ad4e" (UID: "cd56f990-cc2c-465c-b17f-7900fc71ad4e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:07:08.696522 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:08.696503    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-tmp-dir\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:07:08.696572 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:08.696515    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-model-cache" (OuterVolumeSpecName: "model-cache") pod "cd56f990-cc2c-465c-b17f-7900fc71ad4e" (UID: "cd56f990-cc2c-465c-b17f-7900fc71ad4e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:07:08.696572 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:08.696529    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-home\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:07:08.698534 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:08.698504    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd56f990-cc2c-465c-b17f-7900fc71ad4e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cd56f990-cc2c-465c-b17f-7900fc71ad4e" (UID: "cd56f990-cc2c-465c-b17f-7900fc71ad4e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 20:07:08.698708 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:08.698592    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-dshm" (OuterVolumeSpecName: "dshm") pod "cd56f990-cc2c-465c-b17f-7900fc71ad4e" (UID: "cd56f990-cc2c-465c-b17f-7900fc71ad4e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:07:08.698708 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:08.698602    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd56f990-cc2c-465c-b17f-7900fc71ad4e-kube-api-access-lxhjx" (OuterVolumeSpecName: "kube-api-access-lxhjx") pod "cd56f990-cc2c-465c-b17f-7900fc71ad4e" (UID: "cd56f990-cc2c-465c-b17f-7900fc71ad4e"). InnerVolumeSpecName "kube-api-access-lxhjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:07:08.797197 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:08.797155    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-dshm\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:07:08.797197 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:08.797190    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd56f990-cc2c-465c-b17f-7900fc71ad4e-tls-certs\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:07:08.797197 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:08.797200    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd56f990-cc2c-465c-b17f-7900fc71ad4e-model-cache\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:07:08.797439 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:08.797209    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxhjx\" (UniqueName: \"kubernetes.io/projected/cd56f990-cc2c-465c-b17f-7900fc71ad4e-kube-api-access-lxhjx\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:07:09.034767 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.034736    2573 generic.go:356] "Generic (PLEG): container finished" podID="cd56f990-cc2c-465c-b17f-7900fc71ad4e" containerID="bac456b9688f073a6e6eec648d6bb587083737784add035e27d960934705ad42" exitCode=0
Jul 13 20:07:09.034767 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.034779    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"
Jul 13 20:07:09.035004 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.034809    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7" event={"ID":"cd56f990-cc2c-465c-b17f-7900fc71ad4e","Type":"ContainerDied","Data":"bac456b9688f073a6e6eec648d6bb587083737784add035e27d960934705ad42"}
Jul 13 20:07:09.035004 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.034856    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7" event={"ID":"cd56f990-cc2c-465c-b17f-7900fc71ad4e","Type":"ContainerDied","Data":"9981f2705503cc2b6247691c772143c331fbaa0425f8118cc67295b24d1d07ed"}
Jul 13 20:07:09.035004 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.034873    2573 scope.go:117] "RemoveContainer" containerID="bac456b9688f073a6e6eec648d6bb587083737784add035e27d960934705ad42"
Jul 13 20:07:09.044956 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.044939    2573 scope.go:117] "RemoveContainer" containerID="bac456b9688f073a6e6eec648d6bb587083737784add035e27d960934705ad42"
Jul 13 20:07:09.045231 ip-10-0-143-91 kubenswrapper[2573]: E0713 20:07:09.045212    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bac456b9688f073a6e6eec648d6bb587083737784add035e27d960934705ad42\": container with ID starting with bac456b9688f073a6e6eec648d6bb587083737784add035e27d960934705ad42 not found: ID does not exist" containerID="bac456b9688f073a6e6eec648d6bb587083737784add035e27d960934705ad42"
Jul 13 20:07:09.045307 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.045246    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bac456b9688f073a6e6eec648d6bb587083737784add035e27d960934705ad42"} err="failed to get container status \"bac456b9688f073a6e6eec648d6bb587083737784add035e27d960934705ad42\": rpc error: code = NotFound desc = could not find container \"bac456b9688f073a6e6eec648d6bb587083737784add035e27d960934705ad42\": container with ID starting with bac456b9688f073a6e6eec648d6bb587083737784add035e27d960934705ad42 not found: ID does not exist"
Jul 13 20:07:09.057566 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.057532    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"]
Jul 13 20:07:09.061316 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.061286    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-452240/auth-test-model-452240-kserve-68c68bb447-xmnw7"]
Jul 13 20:07:09.331853 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.331753    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp" podUID="93b9a5e5-775e-45bc-bd9c-e058697fabec" containerName="main" containerID="cri-o://1e7d1274d7e77073dcfa6bd22f13a5bf1fdfcdded599db8a047b2ce90cc0ebe3" gracePeriod=45
Jul 13 20:07:09.592078 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.592010    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:07:09.706064 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.706024    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjcfn\" (UniqueName: \"kubernetes.io/projected/93b9a5e5-775e-45bc-bd9c-e058697fabec-kube-api-access-mjcfn\") pod \"93b9a5e5-775e-45bc-bd9c-e058697fabec\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") "
Jul 13 20:07:09.706248 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.706086    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-tmp-dir\") pod \"93b9a5e5-775e-45bc-bd9c-e058697fabec\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") "
Jul 13 20:07:09.706248 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.706207    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-home\") pod \"93b9a5e5-775e-45bc-bd9c-e058697fabec\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") "
Jul 13 20:07:09.706390 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.706278    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93b9a5e5-775e-45bc-bd9c-e058697fabec-tls-certs\") pod \"93b9a5e5-775e-45bc-bd9c-e058697fabec\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") "
Jul 13 20:07:09.706390 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.706311    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-dshm\") pod \"93b9a5e5-775e-45bc-bd9c-e058697fabec\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") "
Jul 13 20:07:09.706390 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.706326    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-model-cache\") pod \"93b9a5e5-775e-45bc-bd9c-e058697fabec\" (UID: \"93b9a5e5-775e-45bc-bd9c-e058697fabec\") "
Jul 13 20:07:09.706390 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.706355    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "93b9a5e5-775e-45bc-bd9c-e058697fabec" (UID: "93b9a5e5-775e-45bc-bd9c-e058697fabec"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:07:09.706588 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.706433    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-home" (OuterVolumeSpecName: "home") pod "93b9a5e5-775e-45bc-bd9c-e058697fabec" (UID: "93b9a5e5-775e-45bc-bd9c-e058697fabec"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:07:09.706588 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.706568    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-model-cache" (OuterVolumeSpecName: "model-cache") pod "93b9a5e5-775e-45bc-bd9c-e058697fabec" (UID: "93b9a5e5-775e-45bc-bd9c-e058697fabec"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:07:09.706713 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.706622    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-tmp-dir\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:07:09.706713 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.706669    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-home\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:07:09.708483 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.708454    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93b9a5e5-775e-45bc-bd9c-e058697fabec-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "93b9a5e5-775e-45bc-bd9c-e058697fabec" (UID: "93b9a5e5-775e-45bc-bd9c-e058697fabec"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 20:07:09.708483 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.708469    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-dshm" (OuterVolumeSpecName: "dshm") pod "93b9a5e5-775e-45bc-bd9c-e058697fabec" (UID: "93b9a5e5-775e-45bc-bd9c-e058697fabec"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:07:09.708483 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.708457    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93b9a5e5-775e-45bc-bd9c-e058697fabec-kube-api-access-mjcfn" (OuterVolumeSpecName: "kube-api-access-mjcfn") pod "93b9a5e5-775e-45bc-bd9c-e058697fabec" (UID: "93b9a5e5-775e-45bc-bd9c-e058697fabec"). InnerVolumeSpecName "kube-api-access-mjcfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:07:09.807112 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.807072    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93b9a5e5-775e-45bc-bd9c-e058697fabec-tls-certs\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:07:09.807112 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.807105    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-dshm\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:07:09.807112 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.807115    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93b9a5e5-775e-45bc-bd9c-e058697fabec-model-cache\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:07:09.807112 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:09.807123    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjcfn\" (UniqueName: \"kubernetes.io/projected/93b9a5e5-775e-45bc-bd9c-e058697fabec-kube-api-access-mjcfn\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:07:10.041768 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:10.041733    2573 generic.go:356] "Generic (PLEG): container finished" podID="93b9a5e5-775e-45bc-bd9c-e058697fabec" containerID="1e7d1274d7e77073dcfa6bd22f13a5bf1fdfcdded599db8a047b2ce90cc0ebe3" exitCode=0
Jul 13 20:07:10.041961 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:10.041778    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"
Jul 13 20:07:10.041961 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:10.041831    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp" event={"ID":"93b9a5e5-775e-45bc-bd9c-e058697fabec","Type":"ContainerDied","Data":"1e7d1274d7e77073dcfa6bd22f13a5bf1fdfcdded599db8a047b2ce90cc0ebe3"}
Jul 13 20:07:10.041961 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:10.041903    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp" event={"ID":"93b9a5e5-775e-45bc-bd9c-e058697fabec","Type":"ContainerDied","Data":"22cd01d3902657b62131056da395516e46b4c1d563ed945a6f3d3e6b3d6d2ea7"}
Jul 13 20:07:10.041961 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:10.041922    2573 scope.go:117] "RemoveContainer" containerID="1e7d1274d7e77073dcfa6bd22f13a5bf1fdfcdded599db8a047b2ce90cc0ebe3"
Jul 13 20:07:10.051365 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:10.051345    2573 scope.go:117] "RemoveContainer" containerID="1e7d1274d7e77073dcfa6bd22f13a5bf1fdfcdded599db8a047b2ce90cc0ebe3"
Jul 13 20:07:10.051666 ip-10-0-143-91 kubenswrapper[2573]: E0713 20:07:10.051621    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e7d1274d7e77073dcfa6bd22f13a5bf1fdfcdded599db8a047b2ce90cc0ebe3\": container with ID starting with 1e7d1274d7e77073dcfa6bd22f13a5bf1fdfcdded599db8a047b2ce90cc0ebe3 not found: ID does not exist" containerID="1e7d1274d7e77073dcfa6bd22f13a5bf1fdfcdded599db8a047b2ce90cc0ebe3"
Jul 13 20:07:10.051726 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:10.051676    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e7d1274d7e77073dcfa6bd22f13a5bf1fdfcdded599db8a047b2ce90cc0ebe3"} err="failed to get container status \"1e7d1274d7e77073dcfa6bd22f13a5bf1fdfcdded599db8a047b2ce90cc0ebe3\": rpc error: code = NotFound desc = could not find container \"1e7d1274d7e77073dcfa6bd22f13a5bf1fdfcdded599db8a047b2ce90cc0ebe3\": container with ID starting with 1e7d1274d7e77073dcfa6bd22f13a5bf1fdfcdded599db8a047b2ce90cc0ebe3 not found: ID does not exist"
Jul 13 20:07:10.063460 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:10.063430    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"]
Jul 13 20:07:10.068813 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:10.068788    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-83eaac/auth-test-model-83eaac-kserve-fd84cddcf-hb5gp"]
Jul 13 20:07:11.048017 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:11.047975    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93b9a5e5-775e-45bc-bd9c-e058697fabec" path="/var/lib/kubelet/pods/93b9a5e5-775e-45bc-bd9c-e058697fabec/volumes"
Jul 13 20:07:11.048567 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:11.048545    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd56f990-cc2c-465c-b17f-7900fc71ad4e" path="/var/lib/kubelet/pods/cd56f990-cc2c-465c-b17f-7900fc71ad4e/volumes"
Jul 13 20:07:12.610200 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.610156    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"]
Jul 13 20:07:12.610599 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.610532    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd56f990-cc2c-465c-b17f-7900fc71ad4e" containerName="main"
Jul 13 20:07:12.610599 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.610543    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd56f990-cc2c-465c-b17f-7900fc71ad4e" containerName="main"
Jul 13 20:07:12.610599 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.610567    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93b9a5e5-775e-45bc-bd9c-e058697fabec" containerName="main"
Jul 13 20:07:12.610599 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.610573    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="93b9a5e5-775e-45bc-bd9c-e058697fabec" containerName="main"
Jul 13 20:07:12.610764 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.610649    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd56f990-cc2c-465c-b17f-7900fc71ad4e" containerName="main"
Jul 13 20:07:12.610764 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.610658    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="93b9a5e5-775e-45bc-bd9c-e058697fabec" containerName="main"
Jul 13 20:07:12.615007 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.614989    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:12.617679 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.617655    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-452240\"/\"kube-root-ca.crt\""
Jul 13 20:07:12.617821 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.617710    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-452240\"/\"openshift-service-ca.crt\""
Jul 13 20:07:12.618686 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.618664    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-452240\"/\"default-dockercfg-jczhj\""
Jul 13 20:07:12.618791 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.618776    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-452240\"/\"sub-test-model-452240-kserve-self-signed-certs\""
Jul 13 20:07:12.622698 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.622675    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"]
Jul 13 20:07:12.733613 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.733574    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4mzb\" (UniqueName: \"kubernetes.io/projected/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-kube-api-access-n4mzb\") pod \"sub-test-model-452240-kserve-d77db6c69-pfwpw\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") " pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:12.733613 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.733613    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-model-cache\") pod \"sub-test-model-452240-kserve-d77db6c69-pfwpw\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") " pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:12.733898 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.733657    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-dshm\") pod \"sub-test-model-452240-kserve-d77db6c69-pfwpw\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") " pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:12.733898 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.733679    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-tls-certs\") pod \"sub-test-model-452240-kserve-d77db6c69-pfwpw\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") " pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:12.733898 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.733872    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-home\") pod \"sub-test-model-452240-kserve-d77db6c69-pfwpw\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") " pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:12.733898 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.733898    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-tmp-dir\") pod \"sub-test-model-452240-kserve-d77db6c69-pfwpw\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") " pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:12.834793 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.834753    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-home\") pod \"sub-test-model-452240-kserve-d77db6c69-pfwpw\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") " pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:12.834793 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.834795    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-tmp-dir\") pod \"sub-test-model-452240-kserve-d77db6c69-pfwpw\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") " pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:12.835030 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.834859    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4mzb\" (UniqueName: \"kubernetes.io/projected/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-kube-api-access-n4mzb\") pod \"sub-test-model-452240-kserve-d77db6c69-pfwpw\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") " pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:12.835030 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.834891    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-model-cache\") pod \"sub-test-model-452240-kserve-d77db6c69-pfwpw\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") " pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:12.835030 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.834925    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-dshm\") pod \"sub-test-model-452240-kserve-d77db6c69-pfwpw\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") " pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:12.835030 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.834944    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-tls-certs\") pod \"sub-test-model-452240-kserve-d77db6c69-pfwpw\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") " pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:12.835292 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.835274    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-tmp-dir\") pod \"sub-test-model-452240-kserve-d77db6c69-pfwpw\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") " pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:12.835427 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.835399    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-model-cache\") pod \"sub-test-model-452240-kserve-d77db6c69-pfwpw\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") " pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:12.835427 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.835415    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-home\") pod \"sub-test-model-452240-kserve-d77db6c69-pfwpw\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") " pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:12.837302 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.837273    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-dshm\") pod \"sub-test-model-452240-kserve-d77db6c69-pfwpw\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") " pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:12.837561 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.837541    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-tls-certs\") pod \"sub-test-model-452240-kserve-d77db6c69-pfwpw\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") " pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:12.843179 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.843160    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4mzb\" (UniqueName: \"kubernetes.io/projected/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-kube-api-access-n4mzb\") pod \"sub-test-model-452240-kserve-d77db6c69-pfwpw\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") " pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:12.925329 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:12.925231    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:13.058484 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:13.058461    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"]
Jul 13 20:07:13.060006 ip-10-0-143-91 kubenswrapper[2573]: W0713 20:07:13.059975    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ceb5b02_602c_444f_b63b_8c39b2b5cb67.slice/crio-9afd0540087cd0ebe2a8301a10ce5852940eea00e1e37ff84bdbccf8f1cb2ab9 WatchSource:0}: Error finding container 9afd0540087cd0ebe2a8301a10ce5852940eea00e1e37ff84bdbccf8f1cb2ab9: Status 404 returned error can't find the container with id 9afd0540087cd0ebe2a8301a10ce5852940eea00e1e37ff84bdbccf8f1cb2ab9
Jul 13 20:07:14.065810 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:14.065768    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw" event={"ID":"6ceb5b02-602c-444f-b63b-8c39b2b5cb67","Type":"ContainerStarted","Data":"aadc2f78e0eff1a651315da5ffa11cae8afd7055bcc2fd09694733d99176e2f4"}
Jul 13 20:07:14.066273 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:14.065819    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw" event={"ID":"6ceb5b02-602c-444f-b63b-8c39b2b5cb67","Type":"ContainerStarted","Data":"9afd0540087cd0ebe2a8301a10ce5852940eea00e1e37ff84bdbccf8f1cb2ab9"}
Jul 13 20:07:14.084287 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:14.084237    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw" podStartSLOduration=2.084221698 podStartE2EDuration="2.084221698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:07:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:07:14.083563536 +0000 UTC m=+2731.597440217" watchObservedRunningTime="2026-07-13 20:07:14.084221698 +0000 UTC m=+2731.598098303"
Jul 13 20:07:22.926370 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:22.926330    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:22.926898 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:22.926392    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:22.938826 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:22.938800    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:23.125525 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:23.125492    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:07:39.905384 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:39.905348    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"]
Jul 13 20:07:39.908972 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:39.908950    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:39.911800 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:39.911777    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-83eaac\"/\"kube-root-ca.crt\""
Jul 13 20:07:39.912763 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:39.912745    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-83eaac\"/\"sub-test-model-83eaac-kserve-self-signed-certs\""
Jul 13 20:07:39.912907 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:39.912888    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-83eaac\"/\"default-dockercfg-v2cbt\""
Jul 13 20:07:39.912957 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:39.912943    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-83eaac\"/\"openshift-service-ca.crt\""
Jul 13 20:07:39.918682 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:39.918657    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"]
Jul 13 20:07:40.080903 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:40.080865    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-home\") pod \"sub-test-model-83eaac-kserve-955cd67cc-z48tj\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") " pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:40.081079 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:40.080940    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-model-cache\") pod \"sub-test-model-83eaac-kserve-955cd67cc-z48tj\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") " pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:40.081079 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:40.080965    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-tmp-dir\") pod \"sub-test-model-83eaac-kserve-955cd67cc-z48tj\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") " pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:40.081079 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:40.080996    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-dshm\") pod \"sub-test-model-83eaac-kserve-955cd67cc-z48tj\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") " pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:40.081079 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:40.081040    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3759ed7-afd7-4e7a-bf0c-e998e7158762-tls-certs\") pod \"sub-test-model-83eaac-kserve-955cd67cc-z48tj\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") " pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:40.081214 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:40.081110    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b4h5\" (UniqueName: \"kubernetes.io/projected/e3759ed7-afd7-4e7a-bf0c-e998e7158762-kube-api-access-2b4h5\") pod \"sub-test-model-83eaac-kserve-955cd67cc-z48tj\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") " pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:40.182295 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:40.182185    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-model-cache\") pod \"sub-test-model-83eaac-kserve-955cd67cc-z48tj\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") " pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:40.182295 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:40.182231    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-tmp-dir\") pod \"sub-test-model-83eaac-kserve-955cd67cc-z48tj\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") " pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:40.182295 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:40.182278    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-dshm\") pod \"sub-test-model-83eaac-kserve-955cd67cc-z48tj\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") " pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:40.182567 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:40.182307    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3759ed7-afd7-4e7a-bf0c-e998e7158762-tls-certs\") pod \"sub-test-model-83eaac-kserve-955cd67cc-z48tj\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") " pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:40.182567 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:40.182333    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b4h5\" (UniqueName: \"kubernetes.io/projected/e3759ed7-afd7-4e7a-bf0c-e998e7158762-kube-api-access-2b4h5\") pod \"sub-test-model-83eaac-kserve-955cd67cc-z48tj\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") " pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:40.182567 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:40.182376    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-home\") pod \"sub-test-model-83eaac-kserve-955cd67cc-z48tj\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") " pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:40.182795 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:40.182697    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-model-cache\") pod \"sub-test-model-83eaac-kserve-955cd67cc-z48tj\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") " pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:40.182930 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:40.182909    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-tmp-dir\") pod \"sub-test-model-83eaac-kserve-955cd67cc-z48tj\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") " pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:40.183020 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:40.182910    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-home\") pod \"sub-test-model-83eaac-kserve-955cd67cc-z48tj\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") " pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:40.184746 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:40.184726    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-dshm\") pod \"sub-test-model-83eaac-kserve-955cd67cc-z48tj\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") " pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:40.185135 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:40.185117    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3759ed7-afd7-4e7a-bf0c-e998e7158762-tls-certs\") pod \"sub-test-model-83eaac-kserve-955cd67cc-z48tj\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") " pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:40.191048 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:40.191024    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b4h5\" (UniqueName: \"kubernetes.io/projected/e3759ed7-afd7-4e7a-bf0c-e998e7158762-kube-api-access-2b4h5\") pod \"sub-test-model-83eaac-kserve-955cd67cc-z48tj\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") " pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:40.219196 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:40.219165    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:40.358091 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:40.358059    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"]
Jul 13 20:07:40.359622 ip-10-0-143-91 kubenswrapper[2573]: W0713 20:07:40.359594    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3759ed7_afd7_4e7a_bf0c_e998e7158762.slice/crio-1f9d5b9d648848cea0a589c8f4debd7f0454f0ad843bbf9f5be9753529bc7809 WatchSource:0}: Error finding container 1f9d5b9d648848cea0a589c8f4debd7f0454f0ad843bbf9f5be9753529bc7809: Status 404 returned error can't find the container with id 1f9d5b9d648848cea0a589c8f4debd7f0454f0ad843bbf9f5be9753529bc7809
Jul 13 20:07:41.204535 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:41.204498    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj" event={"ID":"e3759ed7-afd7-4e7a-bf0c-e998e7158762","Type":"ContainerStarted","Data":"c2cc93c16d6268964637b08af8a76dfe6d74109c468e0bd37d404ea7598c2d97"}
Jul 13 20:07:41.204971 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:41.204547    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj" event={"ID":"e3759ed7-afd7-4e7a-bf0c-e998e7158762","Type":"ContainerStarted","Data":"1f9d5b9d648848cea0a589c8f4debd7f0454f0ad843bbf9f5be9753529bc7809"}
Jul 13 20:07:41.228863 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:41.224363    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj" podStartSLOduration=2.224335996 podStartE2EDuration="2.224335996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:07:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:07:41.221902445 +0000 UTC m=+2758.735779068" watchObservedRunningTime="2026-07-13 20:07:41.224335996 +0000 UTC m=+2758.738212603"
Jul 13 20:07:50.220050 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:50.220012    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:50.220519 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:50.220073    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:50.232679 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:50.232631    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:07:50.266756 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:07:50.266724    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:08:27.518947 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:27.518909    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"]
Jul 13 20:08:28.510072 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:28.510036    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"]
Jul 13 20:08:42.009832 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.009792    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"]
Jul 13 20:08:42.013495 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.013474    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:42.016314 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.016282    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-452240\"/\"rate-test-model-452240-kserve-self-signed-certs\""
Jul 13 20:08:42.027902 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.027873    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"]
Jul 13 20:08:42.044240 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.044207    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-tmp-dir\") pod \"rate-test-model-452240-kserve-7ddc79f795-n88zl\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") " pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:42.044427 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.044247    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwnhl\" (UniqueName: \"kubernetes.io/projected/68fa0688-aed8-4578-8ad3-1de9a02cbd77-kube-api-access-pwnhl\") pod \"rate-test-model-452240-kserve-7ddc79f795-n88zl\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") " pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:42.044427 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.044278    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-dshm\") pod \"rate-test-model-452240-kserve-7ddc79f795-n88zl\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") " pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:42.044427 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.044304    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-home\") pod \"rate-test-model-452240-kserve-7ddc79f795-n88zl\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") " pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:42.044427 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.044321    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-model-cache\") pod \"rate-test-model-452240-kserve-7ddc79f795-n88zl\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") " pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:42.044427 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.044391    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68fa0688-aed8-4578-8ad3-1de9a02cbd77-tls-certs\") pod \"rate-test-model-452240-kserve-7ddc79f795-n88zl\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") " pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:42.145300 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.145261    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwnhl\" (UniqueName: \"kubernetes.io/projected/68fa0688-aed8-4578-8ad3-1de9a02cbd77-kube-api-access-pwnhl\") pod \"rate-test-model-452240-kserve-7ddc79f795-n88zl\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") " pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:42.145509 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.145316    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-dshm\") pod \"rate-test-model-452240-kserve-7ddc79f795-n88zl\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") " pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:42.145509 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.145350    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-home\") pod \"rate-test-model-452240-kserve-7ddc79f795-n88zl\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") " pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:42.145509 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.145369    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-model-cache\") pod \"rate-test-model-452240-kserve-7ddc79f795-n88zl\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") " pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:42.145509 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.145400    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68fa0688-aed8-4578-8ad3-1de9a02cbd77-tls-certs\") pod \"rate-test-model-452240-kserve-7ddc79f795-n88zl\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") " pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:42.145509 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.145447    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-tmp-dir\") pod \"rate-test-model-452240-kserve-7ddc79f795-n88zl\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") " pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:42.145875 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.145852    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-home\") pod \"rate-test-model-452240-kserve-7ddc79f795-n88zl\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") " pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:42.145962 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.145880    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-tmp-dir\") pod \"rate-test-model-452240-kserve-7ddc79f795-n88zl\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") " pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:42.145962 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.145907    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-model-cache\") pod \"rate-test-model-452240-kserve-7ddc79f795-n88zl\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") " pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:42.147777 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.147754    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-dshm\") pod \"rate-test-model-452240-kserve-7ddc79f795-n88zl\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") " pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:42.148099 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.148079    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68fa0688-aed8-4578-8ad3-1de9a02cbd77-tls-certs\") pod \"rate-test-model-452240-kserve-7ddc79f795-n88zl\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") " pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:42.154320 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.154293    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwnhl\" (UniqueName: \"kubernetes.io/projected/68fa0688-aed8-4578-8ad3-1de9a02cbd77-kube-api-access-pwnhl\") pod \"rate-test-model-452240-kserve-7ddc79f795-n88zl\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") " pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:42.322966 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.322867    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:42.456207 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.456182    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"]
Jul 13 20:08:42.458122 ip-10-0-143-91 kubenswrapper[2573]: W0713 20:08:42.458090    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68fa0688_aed8_4578_8ad3_1de9a02cbd77.slice/crio-66e079fcc62b25836521e28c9c789d9d7ad3d31885b04f98ad9e56c7387eed04 WatchSource:0}: Error finding container 66e079fcc62b25836521e28c9c789d9d7ad3d31885b04f98ad9e56c7387eed04: Status 404 returned error can't find the container with id 66e079fcc62b25836521e28c9c789d9d7ad3d31885b04f98ad9e56c7387eed04
Jul 13 20:08:42.527943 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.527907    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl" event={"ID":"68fa0688-aed8-4578-8ad3-1de9a02cbd77","Type":"ContainerStarted","Data":"89985dc7333bdd5381a8892a30f2ad1e31cca4ca1e6d16a0eca4def197848b4b"}
Jul 13 20:08:42.528117 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.527955    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl" event={"ID":"68fa0688-aed8-4578-8ad3-1de9a02cbd77","Type":"ContainerStarted","Data":"66e079fcc62b25836521e28c9c789d9d7ad3d31885b04f98ad9e56c7387eed04"}
Jul 13 20:08:42.539557 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.539525    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw" podUID="6ceb5b02-602c-444f-b63b-8c39b2b5cb67" containerName="main" containerID="cri-o://aadc2f78e0eff1a651315da5ffa11cae8afd7055bcc2fd09694733d99176e2f4" gracePeriod=45
Jul 13 20:08:42.546081 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.546031    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl" podStartSLOduration=1.546015899 podStartE2EDuration="1.546015899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:08:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:08:42.543860396 +0000 UTC m=+2820.057737021" watchObservedRunningTime="2026-07-13 20:08:42.546015899 +0000 UTC m=+2820.059892505"
Jul 13 20:08:42.790087 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.790057    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:08:42.851951 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.851912    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4mzb\" (UniqueName: \"kubernetes.io/projected/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-kube-api-access-n4mzb\") pod \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") "
Jul 13 20:08:42.852149 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.851964    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-tmp-dir\") pod \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") "
Jul 13 20:08:42.852149 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.852006    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-model-cache\") pod \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") "
Jul 13 20:08:42.852149 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.852056    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-home\") pod \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") "
Jul 13 20:08:42.852149 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.852074    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-dshm\") pod \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") "
Jul 13 20:08:42.852149 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.852094    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-tls-certs\") pod \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\" (UID: \"6ceb5b02-602c-444f-b63b-8c39b2b5cb67\") "
Jul 13 20:08:42.852407 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.852261    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-home" (OuterVolumeSpecName: "home") pod "6ceb5b02-602c-444f-b63b-8c39b2b5cb67" (UID: "6ceb5b02-602c-444f-b63b-8c39b2b5cb67"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:08:42.852407 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.852292    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-model-cache" (OuterVolumeSpecName: "model-cache") pod "6ceb5b02-602c-444f-b63b-8c39b2b5cb67" (UID: "6ceb5b02-602c-444f-b63b-8c39b2b5cb67"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:08:42.852407 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.852305    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6ceb5b02-602c-444f-b63b-8c39b2b5cb67" (UID: "6ceb5b02-602c-444f-b63b-8c39b2b5cb67"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:08:42.852407 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.852386    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-model-cache\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:08:42.852407 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.852401    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-home\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:08:42.852407 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.852410    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-tmp-dir\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:08:42.854160 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.854138    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-kube-api-access-n4mzb" (OuterVolumeSpecName: "kube-api-access-n4mzb") pod "6ceb5b02-602c-444f-b63b-8c39b2b5cb67" (UID: "6ceb5b02-602c-444f-b63b-8c39b2b5cb67"). InnerVolumeSpecName "kube-api-access-n4mzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:08:42.854160 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.854145    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6ceb5b02-602c-444f-b63b-8c39b2b5cb67" (UID: "6ceb5b02-602c-444f-b63b-8c39b2b5cb67"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 20:08:42.854264 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.854177    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-dshm" (OuterVolumeSpecName: "dshm") pod "6ceb5b02-602c-444f-b63b-8c39b2b5cb67" (UID: "6ceb5b02-602c-444f-b63b-8c39b2b5cb67"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:08:42.952981 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.952941    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-dshm\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:08:42.952981 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.952979    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-tls-certs\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:08:42.952981 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:42.952990    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4mzb\" (UniqueName: \"kubernetes.io/projected/6ceb5b02-602c-444f-b63b-8c39b2b5cb67-kube-api-access-n4mzb\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:08:43.532079 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.532018    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj" podUID="e3759ed7-afd7-4e7a-bf0c-e998e7158762" containerName="main" containerID="cri-o://c2cc93c16d6268964637b08af8a76dfe6d74109c468e0bd37d404ea7598c2d97" gracePeriod=45
Jul 13 20:08:43.534555 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.534527    2573 generic.go:356] "Generic (PLEG): container finished" podID="6ceb5b02-602c-444f-b63b-8c39b2b5cb67" containerID="aadc2f78e0eff1a651315da5ffa11cae8afd7055bcc2fd09694733d99176e2f4" exitCode=0
Jul 13 20:08:43.534698 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.534618    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"
Jul 13 20:08:43.534806 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.534621    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw" event={"ID":"6ceb5b02-602c-444f-b63b-8c39b2b5cb67","Type":"ContainerDied","Data":"aadc2f78e0eff1a651315da5ffa11cae8afd7055bcc2fd09694733d99176e2f4"}
Jul 13 20:08:43.534870 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.534830    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw" event={"ID":"6ceb5b02-602c-444f-b63b-8c39b2b5cb67","Type":"ContainerDied","Data":"9afd0540087cd0ebe2a8301a10ce5852940eea00e1e37ff84bdbccf8f1cb2ab9"}
Jul 13 20:08:43.534870 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.534863    2573 scope.go:117] "RemoveContainer" containerID="aadc2f78e0eff1a651315da5ffa11cae8afd7055bcc2fd09694733d99176e2f4"
Jul 13 20:08:43.544039 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.544021    2573 scope.go:117] "RemoveContainer" containerID="aadc2f78e0eff1a651315da5ffa11cae8afd7055bcc2fd09694733d99176e2f4"
Jul 13 20:08:43.544286 ip-10-0-143-91 kubenswrapper[2573]: E0713 20:08:43.544269    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aadc2f78e0eff1a651315da5ffa11cae8afd7055bcc2fd09694733d99176e2f4\": container with ID starting with aadc2f78e0eff1a651315da5ffa11cae8afd7055bcc2fd09694733d99176e2f4 not found: ID does not exist" containerID="aadc2f78e0eff1a651315da5ffa11cae8afd7055bcc2fd09694733d99176e2f4"
Jul 13 20:08:43.544345 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.544294    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aadc2f78e0eff1a651315da5ffa11cae8afd7055bcc2fd09694733d99176e2f4"} err="failed to get container status \"aadc2f78e0eff1a651315da5ffa11cae8afd7055bcc2fd09694733d99176e2f4\": rpc error: code = NotFound desc = could not find container \"aadc2f78e0eff1a651315da5ffa11cae8afd7055bcc2fd09694733d99176e2f4\": container with ID starting with aadc2f78e0eff1a651315da5ffa11cae8afd7055bcc2fd09694733d99176e2f4 not found: ID does not exist"
Jul 13 20:08:43.551276 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.551250    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"]
Jul 13 20:08:43.557891 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.557864    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-452240/sub-test-model-452240-kserve-d77db6c69-pfwpw"]
Jul 13 20:08:43.779164 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.779137    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:08:43.860976 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.860890    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2b4h5\" (UniqueName: \"kubernetes.io/projected/e3759ed7-afd7-4e7a-bf0c-e998e7158762-kube-api-access-2b4h5\") pod \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") "
Jul 13 20:08:43.860976 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.860932    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3759ed7-afd7-4e7a-bf0c-e998e7158762-tls-certs\") pod \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") "
Jul 13 20:08:43.861221 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.861057    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-home\") pod \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") "
Jul 13 20:08:43.861221 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.861098    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-dshm\") pod \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") "
Jul 13 20:08:43.861221 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.861133    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-model-cache\") pod \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") "
Jul 13 20:08:43.861221 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.861159    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-tmp-dir\") pod \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\" (UID: \"e3759ed7-afd7-4e7a-bf0c-e998e7158762\") "
Jul 13 20:08:43.861420 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.861336    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-home" (OuterVolumeSpecName: "home") pod "e3759ed7-afd7-4e7a-bf0c-e998e7158762" (UID: "e3759ed7-afd7-4e7a-bf0c-e998e7158762"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:08:43.861470 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.861423    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-model-cache" (OuterVolumeSpecName: "model-cache") pod "e3759ed7-afd7-4e7a-bf0c-e998e7158762" (UID: "e3759ed7-afd7-4e7a-bf0c-e998e7158762"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:08:43.861522 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.861486    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e3759ed7-afd7-4e7a-bf0c-e998e7158762" (UID: "e3759ed7-afd7-4e7a-bf0c-e998e7158762"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:08:43.861522 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.861512    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-home\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:08:43.863199 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.863175    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3759ed7-afd7-4e7a-bf0c-e998e7158762-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e3759ed7-afd7-4e7a-bf0c-e998e7158762" (UID: "e3759ed7-afd7-4e7a-bf0c-e998e7158762"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 20:08:43.863199 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.863177    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3759ed7-afd7-4e7a-bf0c-e998e7158762-kube-api-access-2b4h5" (OuterVolumeSpecName: "kube-api-access-2b4h5") pod "e3759ed7-afd7-4e7a-bf0c-e998e7158762" (UID: "e3759ed7-afd7-4e7a-bf0c-e998e7158762"). InnerVolumeSpecName "kube-api-access-2b4h5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:08:43.863356 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.863232    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-dshm" (OuterVolumeSpecName: "dshm") pod "e3759ed7-afd7-4e7a-bf0c-e998e7158762" (UID: "e3759ed7-afd7-4e7a-bf0c-e998e7158762"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:08:43.962125 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.962088    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-dshm\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:08:43.962125 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.962118    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-model-cache\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:08:43.962125 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.962128    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3759ed7-afd7-4e7a-bf0c-e998e7158762-tmp-dir\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:08:43.962366 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.962139    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2b4h5\" (UniqueName: \"kubernetes.io/projected/e3759ed7-afd7-4e7a-bf0c-e998e7158762-kube-api-access-2b4h5\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:08:43.962366 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:43.962149    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3759ed7-afd7-4e7a-bf0c-e998e7158762-tls-certs\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:08:44.541351 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:44.541314    2573 generic.go:356] "Generic (PLEG): container finished" podID="e3759ed7-afd7-4e7a-bf0c-e998e7158762" containerID="c2cc93c16d6268964637b08af8a76dfe6d74109c468e0bd37d404ea7598c2d97" exitCode=0
Jul 13 20:08:44.541851 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:44.541361    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"
Jul 13 20:08:44.541851 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:44.541406    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj" event={"ID":"e3759ed7-afd7-4e7a-bf0c-e998e7158762","Type":"ContainerDied","Data":"c2cc93c16d6268964637b08af8a76dfe6d74109c468e0bd37d404ea7598c2d97"}
Jul 13 20:08:44.541851 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:44.541459    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj" event={"ID":"e3759ed7-afd7-4e7a-bf0c-e998e7158762","Type":"ContainerDied","Data":"1f9d5b9d648848cea0a589c8f4debd7f0454f0ad843bbf9f5be9753529bc7809"}
Jul 13 20:08:44.541851 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:44.541486    2573 scope.go:117] "RemoveContainer" containerID="c2cc93c16d6268964637b08af8a76dfe6d74109c468e0bd37d404ea7598c2d97"
Jul 13 20:08:44.551075 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:44.551058    2573 scope.go:117] "RemoveContainer" containerID="c2cc93c16d6268964637b08af8a76dfe6d74109c468e0bd37d404ea7598c2d97"
Jul 13 20:08:44.551311 ip-10-0-143-91 kubenswrapper[2573]: E0713 20:08:44.551293    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2cc93c16d6268964637b08af8a76dfe6d74109c468e0bd37d404ea7598c2d97\": container with ID starting with c2cc93c16d6268964637b08af8a76dfe6d74109c468e0bd37d404ea7598c2d97 not found: ID does not exist" containerID="c2cc93c16d6268964637b08af8a76dfe6d74109c468e0bd37d404ea7598c2d97"
Jul 13 20:08:44.551353 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:44.551319    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2cc93c16d6268964637b08af8a76dfe6d74109c468e0bd37d404ea7598c2d97"} err="failed to get container status \"c2cc93c16d6268964637b08af8a76dfe6d74109c468e0bd37d404ea7598c2d97\": rpc error: code = NotFound desc = could not find container \"c2cc93c16d6268964637b08af8a76dfe6d74109c468e0bd37d404ea7598c2d97\": container with ID starting with c2cc93c16d6268964637b08af8a76dfe6d74109c468e0bd37d404ea7598c2d97 not found: ID does not exist"
Jul 13 20:08:44.561977 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:44.561950    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"]
Jul 13 20:08:44.565578 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:44.565552    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-83eaac/sub-test-model-83eaac-kserve-955cd67cc-z48tj"]
Jul 13 20:08:45.048468 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:45.048437    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ceb5b02-602c-444f-b63b-8c39b2b5cb67" path="/var/lib/kubelet/pods/6ceb5b02-602c-444f-b63b-8c39b2b5cb67/volumes"
Jul 13 20:08:45.048853 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:45.048839    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3759ed7-afd7-4e7a-bf0c-e998e7158762" path="/var/lib/kubelet/pods/e3759ed7-afd7-4e7a-bf0c-e998e7158762/volumes"
Jul 13 20:08:52.323178 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:52.323144    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:52.323565 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:52.323218    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:52.336163 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:52.336136    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:08:52.598670 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:08:52.598557    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:09:15.706775 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.706734    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"]
Jul 13 20:09:15.707268 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.707118    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3759ed7-afd7-4e7a-bf0c-e998e7158762" containerName="main"
Jul 13 20:09:15.707268 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.707135    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3759ed7-afd7-4e7a-bf0c-e998e7158762" containerName="main"
Jul 13 20:09:15.707268 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.707152    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ceb5b02-602c-444f-b63b-8c39b2b5cb67" containerName="main"
Jul 13 20:09:15.707268 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.707160    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ceb5b02-602c-444f-b63b-8c39b2b5cb67" containerName="main"
Jul 13 20:09:15.707415 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.707303    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ceb5b02-602c-444f-b63b-8c39b2b5cb67" containerName="main"
Jul 13 20:09:15.707415 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.707323    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3759ed7-afd7-4e7a-bf0c-e998e7158762" containerName="main"
Jul 13 20:09:15.710913 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.710897    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:15.713667 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.713614    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-83eaac\"/\"openshift-service-ca.crt\""
Jul 13 20:09:15.714684 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.714655    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-83eaac\"/\"kube-root-ca.crt\""
Jul 13 20:09:15.714784 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.714746    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-83eaac\"/\"rate-test-model-83eaac-kserve-self-signed-certs\""
Jul 13 20:09:15.714900 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.714879    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-83eaac\"/\"default-dockercfg-v2cbt\""
Jul 13 20:09:15.720211 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.720184    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"]
Jul 13 20:09:15.865116 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.865068    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-tmp-dir\") pod \"rate-test-model-83eaac-kserve-795f46c6d7-5zwzv\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") " pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:15.865311 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.865139    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-home\") pod \"rate-test-model-83eaac-kserve-795f46c6d7-5zwzv\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") " pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:15.865311 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.865185    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-dshm\") pod \"rate-test-model-83eaac-kserve-795f46c6d7-5zwzv\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") " pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:15.865311 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.865294    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5672bada-ac53-4322-8b38-f33c0ee3af29-tls-certs\") pod \"rate-test-model-83eaac-kserve-795f46c6d7-5zwzv\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") " pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:15.865423 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.865327    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mzn5\" (UniqueName: \"kubernetes.io/projected/5672bada-ac53-4322-8b38-f33c0ee3af29-kube-api-access-6mzn5\") pod \"rate-test-model-83eaac-kserve-795f46c6d7-5zwzv\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") " pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:15.865423 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.865353    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-model-cache\") pod \"rate-test-model-83eaac-kserve-795f46c6d7-5zwzv\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") " pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:15.966211 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.966110    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5672bada-ac53-4322-8b38-f33c0ee3af29-tls-certs\") pod \"rate-test-model-83eaac-kserve-795f46c6d7-5zwzv\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") " pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:15.966211 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.966159    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mzn5\" (UniqueName: \"kubernetes.io/projected/5672bada-ac53-4322-8b38-f33c0ee3af29-kube-api-access-6mzn5\") pod \"rate-test-model-83eaac-kserve-795f46c6d7-5zwzv\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") " pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:15.966211 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.966185    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-model-cache\") pod \"rate-test-model-83eaac-kserve-795f46c6d7-5zwzv\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") " pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:15.966211 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.966211    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-tmp-dir\") pod \"rate-test-model-83eaac-kserve-795f46c6d7-5zwzv\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") " pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:15.966534 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.966240    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-home\") pod \"rate-test-model-83eaac-kserve-795f46c6d7-5zwzv\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") " pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:15.966534 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.966285    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-dshm\") pod \"rate-test-model-83eaac-kserve-795f46c6d7-5zwzv\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") " pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:15.966707 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.966683    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-model-cache\") pod \"rate-test-model-83eaac-kserve-795f46c6d7-5zwzv\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") " pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:15.966786 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.966750    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-home\") pod \"rate-test-model-83eaac-kserve-795f46c6d7-5zwzv\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") " pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:15.966847 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.966822    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-tmp-dir\") pod \"rate-test-model-83eaac-kserve-795f46c6d7-5zwzv\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") " pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:15.968547 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.968528    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-dshm\") pod \"rate-test-model-83eaac-kserve-795f46c6d7-5zwzv\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") " pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:15.968839 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.968819    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5672bada-ac53-4322-8b38-f33c0ee3af29-tls-certs\") pod \"rate-test-model-83eaac-kserve-795f46c6d7-5zwzv\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") " pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:15.975437 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:15.975414    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mzn5\" (UniqueName: \"kubernetes.io/projected/5672bada-ac53-4322-8b38-f33c0ee3af29-kube-api-access-6mzn5\") pod \"rate-test-model-83eaac-kserve-795f46c6d7-5zwzv\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") " pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:16.020840 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:16.020797    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:16.161353 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:16.161320    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"]
Jul 13 20:09:16.163925 ip-10-0-143-91 kubenswrapper[2573]: W0713 20:09:16.163896    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5672bada_ac53_4322_8b38_f33c0ee3af29.slice/crio-48ff6c31f98cce608d0f63d93b187253285d9457d1d14381d837429f358b9e9d WatchSource:0}: Error finding container 48ff6c31f98cce608d0f63d93b187253285d9457d1d14381d837429f358b9e9d: Status 404 returned error can't find the container with id 48ff6c31f98cce608d0f63d93b187253285d9457d1d14381d837429f358b9e9d
Jul 13 20:09:16.716537 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:16.716494    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv" event={"ID":"5672bada-ac53-4322-8b38-f33c0ee3af29","Type":"ContainerStarted","Data":"2732fe71ffa2ae73d12e659f8bc36f604047618510c476a7157c91c53f8772f7"}
Jul 13 20:09:16.716537 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:16.716542    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv" event={"ID":"5672bada-ac53-4322-8b38-f33c0ee3af29","Type":"ContainerStarted","Data":"48ff6c31f98cce608d0f63d93b187253285d9457d1d14381d837429f358b9e9d"}
Jul 13 20:09:16.737572 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:16.737515    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv" podStartSLOduration=1.73749749 podStartE2EDuration="1.73749749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:09:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:09:16.736603357 +0000 UTC m=+2854.250479990" watchObservedRunningTime="2026-07-13 20:09:16.73749749 +0000 UTC m=+2854.251374097"
Jul 13 20:09:26.020973 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:26.020934    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:26.021381 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:26.021014    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:26.033490 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:26.033460    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:09:26.779495 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:09:26.779465    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:10:04.306798 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:04.306759    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"]
Jul 13 20:10:05.311189 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:05.311154    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"]
Jul 13 20:10:19.326770 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:19.326726    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl" podUID="68fa0688-aed8-4578-8ad3-1de9a02cbd77" containerName="main" containerID="cri-o://89985dc7333bdd5381a8892a30f2ad1e31cca4ca1e6d16a0eca4def197848b4b" gracePeriod=45
Jul 13 20:10:19.587049 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:19.586983    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:10:19.624945 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:19.624911    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-model-cache\") pod \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") "
Jul 13 20:10:19.624945 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:19.624953    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68fa0688-aed8-4578-8ad3-1de9a02cbd77-tls-certs\") pod \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") "
Jul 13 20:10:19.625189 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:19.624985    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-tmp-dir\") pod \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") "
Jul 13 20:10:19.625189 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:19.625020    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-home\") pod \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") "
Jul 13 20:10:19.625189 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:19.625083    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwnhl\" (UniqueName: \"kubernetes.io/projected/68fa0688-aed8-4578-8ad3-1de9a02cbd77-kube-api-access-pwnhl\") pod \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") "
Jul 13 20:10:19.625189 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:19.625162    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-dshm\") pod \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\" (UID: \"68fa0688-aed8-4578-8ad3-1de9a02cbd77\") "
Jul 13 20:10:19.625414 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:19.625244    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-model-cache" (OuterVolumeSpecName: "model-cache") pod "68fa0688-aed8-4578-8ad3-1de9a02cbd77" (UID: "68fa0688-aed8-4578-8ad3-1de9a02cbd77"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:10:19.625414 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:19.625313    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-home" (OuterVolumeSpecName: "home") pod "68fa0688-aed8-4578-8ad3-1de9a02cbd77" (UID: "68fa0688-aed8-4578-8ad3-1de9a02cbd77"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:10:19.625521 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:19.625482    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-home\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:10:19.625521 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:19.625502    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-model-cache\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:10:19.625625 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:19.625578    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "68fa0688-aed8-4578-8ad3-1de9a02cbd77" (UID: "68fa0688-aed8-4578-8ad3-1de9a02cbd77"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:10:19.627793 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:19.627766    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68fa0688-aed8-4578-8ad3-1de9a02cbd77-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "68fa0688-aed8-4578-8ad3-1de9a02cbd77" (UID: "68fa0688-aed8-4578-8ad3-1de9a02cbd77"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 20:10:19.627793 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:19.627786    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-dshm" (OuterVolumeSpecName: "dshm") pod "68fa0688-aed8-4578-8ad3-1de9a02cbd77" (UID: "68fa0688-aed8-4578-8ad3-1de9a02cbd77"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:10:19.628003 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:19.627911    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68fa0688-aed8-4578-8ad3-1de9a02cbd77-kube-api-access-pwnhl" (OuterVolumeSpecName: "kube-api-access-pwnhl") pod "68fa0688-aed8-4578-8ad3-1de9a02cbd77" (UID: "68fa0688-aed8-4578-8ad3-1de9a02cbd77"). InnerVolumeSpecName "kube-api-access-pwnhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:10:19.726877 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:19.726830    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-dshm\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:10:19.726877 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:19.726873    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68fa0688-aed8-4578-8ad3-1de9a02cbd77-tls-certs\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:10:19.726877 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:19.726887    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68fa0688-aed8-4578-8ad3-1de9a02cbd77-tmp-dir\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:10:19.727137 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:19.726900    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwnhl\" (UniqueName: \"kubernetes.io/projected/68fa0688-aed8-4578-8ad3-1de9a02cbd77-kube-api-access-pwnhl\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:10:20.051310 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.051270    2573 generic.go:356] "Generic (PLEG): container finished" podID="68fa0688-aed8-4578-8ad3-1de9a02cbd77" containerID="89985dc7333bdd5381a8892a30f2ad1e31cca4ca1e6d16a0eca4def197848b4b" exitCode=0
Jul 13 20:10:20.051310 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.051316    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"
Jul 13 20:10:20.051569 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.051353    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl" event={"ID":"68fa0688-aed8-4578-8ad3-1de9a02cbd77","Type":"ContainerDied","Data":"89985dc7333bdd5381a8892a30f2ad1e31cca4ca1e6d16a0eca4def197848b4b"}
Jul 13 20:10:20.051569 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.051401    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl" event={"ID":"68fa0688-aed8-4578-8ad3-1de9a02cbd77","Type":"ContainerDied","Data":"66e079fcc62b25836521e28c9c789d9d7ad3d31885b04f98ad9e56c7387eed04"}
Jul 13 20:10:20.051569 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.051418    2573 scope.go:117] "RemoveContainer" containerID="89985dc7333bdd5381a8892a30f2ad1e31cca4ca1e6d16a0eca4def197848b4b"
Jul 13 20:10:20.064014 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.062227    2573 scope.go:117] "RemoveContainer" containerID="89985dc7333bdd5381a8892a30f2ad1e31cca4ca1e6d16a0eca4def197848b4b"
Jul 13 20:10:20.064014 ip-10-0-143-91 kubenswrapper[2573]: E0713 20:10:20.062593    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89985dc7333bdd5381a8892a30f2ad1e31cca4ca1e6d16a0eca4def197848b4b\": container with ID starting with 89985dc7333bdd5381a8892a30f2ad1e31cca4ca1e6d16a0eca4def197848b4b not found: ID does not exist" containerID="89985dc7333bdd5381a8892a30f2ad1e31cca4ca1e6d16a0eca4def197848b4b"
Jul 13 20:10:20.064014 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.062625    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89985dc7333bdd5381a8892a30f2ad1e31cca4ca1e6d16a0eca4def197848b4b"} err="failed to get container status \"89985dc7333bdd5381a8892a30f2ad1e31cca4ca1e6d16a0eca4def197848b4b\": rpc error: code = NotFound desc = could not find container \"89985dc7333bdd5381a8892a30f2ad1e31cca4ca1e6d16a0eca4def197848b4b\": container with ID starting with 89985dc7333bdd5381a8892a30f2ad1e31cca4ca1e6d16a0eca4def197848b4b not found: ID does not exist"
Jul 13 20:10:20.076707 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.076676    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"]
Jul 13 20:10:20.080401 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.080374    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-452240/rate-test-model-452240-kserve-7ddc79f795-n88zl"]
Jul 13 20:10:20.331493 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.331389    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv" podUID="5672bada-ac53-4322-8b38-f33c0ee3af29" containerName="main" containerID="cri-o://2732fe71ffa2ae73d12e659f8bc36f604047618510c476a7157c91c53f8772f7" gracePeriod=45
Jul 13 20:10:20.589304 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.589234    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:10:20.637459 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.637406    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5672bada-ac53-4322-8b38-f33c0ee3af29-tls-certs\") pod \"5672bada-ac53-4322-8b38-f33c0ee3af29\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") "
Jul 13 20:10:20.637710 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.637479    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-home\") pod \"5672bada-ac53-4322-8b38-f33c0ee3af29\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") "
Jul 13 20:10:20.637710 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.637512    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mzn5\" (UniqueName: \"kubernetes.io/projected/5672bada-ac53-4322-8b38-f33c0ee3af29-kube-api-access-6mzn5\") pod \"5672bada-ac53-4322-8b38-f33c0ee3af29\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") "
Jul 13 20:10:20.637710 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.637626    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-dshm\") pod \"5672bada-ac53-4322-8b38-f33c0ee3af29\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") "
Jul 13 20:10:20.637710 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.637677    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-tmp-dir\") pod \"5672bada-ac53-4322-8b38-f33c0ee3af29\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") "
Jul 13 20:10:20.637710 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.637708    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-model-cache\") pod \"5672bada-ac53-4322-8b38-f33c0ee3af29\" (UID: \"5672bada-ac53-4322-8b38-f33c0ee3af29\") "
Jul 13 20:10:20.637979 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.637775    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-home" (OuterVolumeSpecName: "home") pod "5672bada-ac53-4322-8b38-f33c0ee3af29" (UID: "5672bada-ac53-4322-8b38-f33c0ee3af29"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:10:20.638038 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.637976    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5672bada-ac53-4322-8b38-f33c0ee3af29" (UID: "5672bada-ac53-4322-8b38-f33c0ee3af29"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:10:20.638038 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.637993    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-home\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:10:20.638175 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.638152    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-model-cache" (OuterVolumeSpecName: "model-cache") pod "5672bada-ac53-4322-8b38-f33c0ee3af29" (UID: "5672bada-ac53-4322-8b38-f33c0ee3af29"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:10:20.640008 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.639980    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-dshm" (OuterVolumeSpecName: "dshm") pod "5672bada-ac53-4322-8b38-f33c0ee3af29" (UID: "5672bada-ac53-4322-8b38-f33c0ee3af29"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:10:20.640280 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.640264    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5672bada-ac53-4322-8b38-f33c0ee3af29-kube-api-access-6mzn5" (OuterVolumeSpecName: "kube-api-access-6mzn5") pod "5672bada-ac53-4322-8b38-f33c0ee3af29" (UID: "5672bada-ac53-4322-8b38-f33c0ee3af29"). InnerVolumeSpecName "kube-api-access-6mzn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:10:20.640368 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.640347    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5672bada-ac53-4322-8b38-f33c0ee3af29-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5672bada-ac53-4322-8b38-f33c0ee3af29" (UID: "5672bada-ac53-4322-8b38-f33c0ee3af29"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 20:10:20.738653 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.738596    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-model-cache\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:10:20.738869 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.738670    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5672bada-ac53-4322-8b38-f33c0ee3af29-tls-certs\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:10:20.738869 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.738686    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mzn5\" (UniqueName: \"kubernetes.io/projected/5672bada-ac53-4322-8b38-f33c0ee3af29-kube-api-access-6mzn5\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:10:20.738869 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.738700    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-dshm\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:10:20.738869 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:20.738733    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5672bada-ac53-4322-8b38-f33c0ee3af29-tmp-dir\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:10:21.048140 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.048104    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68fa0688-aed8-4578-8ad3-1de9a02cbd77" path="/var/lib/kubelet/pods/68fa0688-aed8-4578-8ad3-1de9a02cbd77/volumes"
Jul 13 20:10:21.058607 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.058578    2573 generic.go:356] "Generic (PLEG): container finished" podID="5672bada-ac53-4322-8b38-f33c0ee3af29" containerID="2732fe71ffa2ae73d12e659f8bc36f604047618510c476a7157c91c53f8772f7" exitCode=0
Jul 13 20:10:21.058751 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.058620    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"
Jul 13 20:10:21.058751 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.058665    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv" event={"ID":"5672bada-ac53-4322-8b38-f33c0ee3af29","Type":"ContainerDied","Data":"2732fe71ffa2ae73d12e659f8bc36f604047618510c476a7157c91c53f8772f7"}
Jul 13 20:10:21.058751 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.058707    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv" event={"ID":"5672bada-ac53-4322-8b38-f33c0ee3af29","Type":"ContainerDied","Data":"48ff6c31f98cce608d0f63d93b187253285d9457d1d14381d837429f358b9e9d"}
Jul 13 20:10:21.058751 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.058719    2573 scope.go:117] "RemoveContainer" containerID="2732fe71ffa2ae73d12e659f8bc36f604047618510c476a7157c91c53f8772f7"
Jul 13 20:10:21.067445 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.067424    2573 scope.go:117] "RemoveContainer" containerID="2732fe71ffa2ae73d12e659f8bc36f604047618510c476a7157c91c53f8772f7"
Jul 13 20:10:21.067741 ip-10-0-143-91 kubenswrapper[2573]: E0713 20:10:21.067713    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2732fe71ffa2ae73d12e659f8bc36f604047618510c476a7157c91c53f8772f7\": container with ID starting with 2732fe71ffa2ae73d12e659f8bc36f604047618510c476a7157c91c53f8772f7 not found: ID does not exist" containerID="2732fe71ffa2ae73d12e659f8bc36f604047618510c476a7157c91c53f8772f7"
Jul 13 20:10:21.067842 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.067748    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2732fe71ffa2ae73d12e659f8bc36f604047618510c476a7157c91c53f8772f7"} err="failed to get container status \"2732fe71ffa2ae73d12e659f8bc36f604047618510c476a7157c91c53f8772f7\": rpc error: code = NotFound desc = could not find container \"2732fe71ffa2ae73d12e659f8bc36f604047618510c476a7157c91c53f8772f7\": container with ID starting with 2732fe71ffa2ae73d12e659f8bc36f604047618510c476a7157c91c53f8772f7 not found: ID does not exist"
Jul 13 20:10:21.081034 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.081009    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"]
Jul 13 20:10:21.086384 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.086355    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-83eaac/rate-test-model-83eaac-kserve-795f46c6d7-5zwzv"]
Jul 13 20:10:21.582196 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.582155    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7702-139685475591200"]
Jul 13 20:10:21.582585 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.582513    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5672bada-ac53-4322-8b38-f33c0ee3af29" containerName="main"
Jul 13 20:10:21.582585 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.582525    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5672bada-ac53-4322-8b38-f33c0ee3af29" containerName="main"
Jul 13 20:10:21.582585 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.582543    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68fa0688-aed8-4578-8ad3-1de9a02cbd77" containerName="main"
Jul 13 20:10:21.582585 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.582548    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="68fa0688-aed8-4578-8ad3-1de9a02cbd77" containerName="main"
Jul 13 20:10:21.582763 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.582612    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5672bada-ac53-4322-8b38-f33c0ee3af29" containerName="main"
Jul 13 20:10:21.582763 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.582620    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="68fa0688-aed8-4578-8ad3-1de9a02cbd77" containerName="main"
Jul 13 20:10:21.585463 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.585430    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475591200"
Jul 13 20:10:21.588275 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.588249    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-rddtv\""
Jul 13 20:10:21.595719 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.595686    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7702-139685475591200"]
Jul 13 20:10:21.646630 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.646593    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9pgb\" (UniqueName: \"kubernetes.io/projected/428a18d2-a6cb-4aaa-8a1b-b2604e8cbd5b-kube-api-access-d9pgb\") pod \"test-curl-7702-139685475591200\" (UID: \"428a18d2-a6cb-4aaa-8a1b-b2604e8cbd5b\") " pod="openshift-ingress/test-curl-7702-139685475591200"
Jul 13 20:10:21.748134 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.748096    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9pgb\" (UniqueName: \"kubernetes.io/projected/428a18d2-a6cb-4aaa-8a1b-b2604e8cbd5b-kube-api-access-d9pgb\") pod \"test-curl-7702-139685475591200\" (UID: \"428a18d2-a6cb-4aaa-8a1b-b2604e8cbd5b\") " pod="openshift-ingress/test-curl-7702-139685475591200"
Jul 13 20:10:21.758176 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.758145    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9pgb\" (UniqueName: \"kubernetes.io/projected/428a18d2-a6cb-4aaa-8a1b-b2604e8cbd5b-kube-api-access-d9pgb\") pod \"test-curl-7702-139685475591200\" (UID: \"428a18d2-a6cb-4aaa-8a1b-b2604e8cbd5b\") " pod="openshift-ingress/test-curl-7702-139685475591200"
Jul 13 20:10:21.895414 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:21.895325    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475591200"
Jul 13 20:10:22.033823 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:22.033794    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7702-139685475591200"]
Jul 13 20:10:22.035747 ip-10-0-143-91 kubenswrapper[2573]: W0713 20:10:22.035716    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod428a18d2_a6cb_4aaa_8a1b_b2604e8cbd5b.slice/crio-53319597b57ac1221843c1e8ba36813f0984aa4b71d72979b2325fac003a8a22 WatchSource:0}: Error finding container 53319597b57ac1221843c1e8ba36813f0984aa4b71d72979b2325fac003a8a22: Status 404 returned error can't find the container with id 53319597b57ac1221843c1e8ba36813f0984aa4b71d72979b2325fac003a8a22
Jul 13 20:10:22.037109 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:22.037078    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 20:10:22.065923 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:22.065884    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7702-139685475591200" event={"ID":"428a18d2-a6cb-4aaa-8a1b-b2604e8cbd5b","Type":"ContainerStarted","Data":"53319597b57ac1221843c1e8ba36813f0984aa4b71d72979b2325fac003a8a22"}
Jul 13 20:10:23.050382 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:23.050298    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5672bada-ac53-4322-8b38-f33c0ee3af29" path="/var/lib/kubelet/pods/5672bada-ac53-4322-8b38-f33c0ee3af29/volumes"
Jul 13 20:10:23.072564 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:23.072530    2573 generic.go:356] "Generic (PLEG): container finished" podID="428a18d2-a6cb-4aaa-8a1b-b2604e8cbd5b" containerID="84bc6f2a4ff82a22eb02632e35406f41cb29eef75711039c4bff198bb6b9a893" exitCode=0
Jul 13 20:10:23.072763 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:23.072620    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7702-139685475591200" event={"ID":"428a18d2-a6cb-4aaa-8a1b-b2604e8cbd5b","Type":"ContainerDied","Data":"84bc6f2a4ff82a22eb02632e35406f41cb29eef75711039c4bff198bb6b9a893"}
Jul 13 20:10:24.223117 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:24.223089    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475591200"
Jul 13 20:10:24.265408 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:24.265373    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7702-139685475591200_428a18d2-a6cb-4aaa-8a1b-b2604e8cbd5b/test-curl-7702-139685475591200/0.log"
Jul 13 20:10:24.269684 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:24.269662    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9pgb\" (UniqueName: \"kubernetes.io/projected/428a18d2-a6cb-4aaa-8a1b-b2604e8cbd5b-kube-api-access-d9pgb\") pod \"428a18d2-a6cb-4aaa-8a1b-b2604e8cbd5b\" (UID: \"428a18d2-a6cb-4aaa-8a1b-b2604e8cbd5b\") "
Jul 13 20:10:24.271755 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:24.271724    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/428a18d2-a6cb-4aaa-8a1b-b2604e8cbd5b-kube-api-access-d9pgb" (OuterVolumeSpecName: "kube-api-access-d9pgb") pod "428a18d2-a6cb-4aaa-8a1b-b2604e8cbd5b" (UID: "428a18d2-a6cb-4aaa-8a1b-b2604e8cbd5b"). InnerVolumeSpecName "kube-api-access-d9pgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:10:24.290763 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:24.290736    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7702-139685475591200"]
Jul 13 20:10:24.294544 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:24.294520    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7702-139685475591200"]
Jul 13 20:10:24.371039 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:24.370946    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9pgb\" (UniqueName: \"kubernetes.io/projected/428a18d2-a6cb-4aaa-8a1b-b2604e8cbd5b-kube-api-access-d9pgb\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:10:24.565082 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:24.565051    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-452240-v4x8z"]
Jul 13 20:10:24.565402 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:24.565390    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="428a18d2-a6cb-4aaa-8a1b-b2604e8cbd5b" containerName="test-curl-7702-139685475591200"
Jul 13 20:10:24.565446 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:24.565404    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="428a18d2-a6cb-4aaa-8a1b-b2604e8cbd5b" containerName="test-curl-7702-139685475591200"
Jul 13 20:10:24.565487 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:24.565472    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="428a18d2-a6cb-4aaa-8a1b-b2604e8cbd5b" containerName="test-curl-7702-139685475591200"
Jul 13 20:10:24.568612 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:24.568592    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-452240-v4x8z"
Jul 13 20:10:24.571443 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:24.571418    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 20:10:24.572616 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:24.572587    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q5n9g\""
Jul 13 20:10:24.580315 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:24.580287    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-452240-v4x8z"]
Jul 13 20:10:24.673131 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:24.673039    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47be563a-cf47-4eb0-b04c-72e673ba4996-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-452240-v4x8z\" (UID: \"47be563a-cf47-4eb0-b04c-72e673ba4996\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-452240-v4x8z"
Jul 13 20:10:24.774550 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:24.774509    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47be563a-cf47-4eb0-b04c-72e673ba4996-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-452240-v4x8z\" (UID: \"47be563a-cf47-4eb0-b04c-72e673ba4996\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-452240-v4x8z"
Jul 13 20:10:24.775266 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:24.775232    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47be563a-cf47-4eb0-b04c-72e673ba4996-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-452240-v4x8z\" (UID: \"47be563a-cf47-4eb0-b04c-72e673ba4996\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-452240-v4x8z"
Jul 13 20:10:24.880196 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:24.880159    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-452240-v4x8z"
Jul 13 20:10:25.018345 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:25.018314    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-452240-v4x8z"]
Jul 13 20:10:25.020523 ip-10-0-143-91 kubenswrapper[2573]: W0713 20:10:25.020493    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47be563a_cf47_4eb0_b04c_72e673ba4996.slice/crio-a934a48e0a897836c544a4b548d7b8e4a51673ef3deb82411c0008d0697092b2 WatchSource:0}: Error finding container a934a48e0a897836c544a4b548d7b8e4a51673ef3deb82411c0008d0697092b2: Status 404 returned error can't find the container with id a934a48e0a897836c544a4b548d7b8e4a51673ef3deb82411c0008d0697092b2
Jul 13 20:10:25.048819 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:25.048787    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="428a18d2-a6cb-4aaa-8a1b-b2604e8cbd5b" path="/var/lib/kubelet/pods/428a18d2-a6cb-4aaa-8a1b-b2604e8cbd5b/volumes"
Jul 13 20:10:25.084820 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:25.084786    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475591200"
Jul 13 20:10:25.084820 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:25.084814    2573 scope.go:117] "RemoveContainer" containerID="84bc6f2a4ff82a22eb02632e35406f41cb29eef75711039c4bff198bb6b9a893"
Jul 13 20:10:25.086329 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:25.086306    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-452240-v4x8z" event={"ID":"47be563a-cf47-4eb0-b04c-72e673ba4996","Type":"ContainerStarted","Data":"a934a48e0a897836c544a4b548d7b8e4a51673ef3deb82411c0008d0697092b2"}
Jul 13 20:10:26.094003 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:26.093966    2573 generic.go:356] "Generic (PLEG): container finished" podID="47be563a-cf47-4eb0-b04c-72e673ba4996" containerID="0551dc8cc2197d7c809465f9be404421c01256b318f9eeabd3a5bb968cd79a6b" exitCode=0
Jul 13 20:10:26.094409 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:26.094021    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-452240-v4x8z" event={"ID":"47be563a-cf47-4eb0-b04c-72e673ba4996","Type":"ContainerDied","Data":"0551dc8cc2197d7c809465f9be404421c01256b318f9eeabd3a5bb968cd79a6b"}
Jul 13 20:10:27.231059 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:27.231035    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-452240-v4x8z"
Jul 13 20:10:27.296746 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:27.296706    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47be563a-cf47-4eb0-b04c-72e673ba4996-maas-api-service-ca\") pod \"47be563a-cf47-4eb0-b04c-72e673ba4996\" (UID: \"47be563a-cf47-4eb0-b04c-72e673ba4996\") "
Jul 13 20:10:27.297122 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:27.297094    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47be563a-cf47-4eb0-b04c-72e673ba4996-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "47be563a-cf47-4eb0-b04c-72e673ba4996" (UID: "47be563a-cf47-4eb0-b04c-72e673ba4996"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 20:10:27.397928 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:27.397824    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47be563a-cf47-4eb0-b04c-72e673ba4996-maas-api-service-ca\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:10:28.105682 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:28.105628    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-452240-v4x8z"
Jul 13 20:10:28.105682 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:28.105668    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-452240-v4x8z" event={"ID":"47be563a-cf47-4eb0-b04c-72e673ba4996","Type":"ContainerDied","Data":"a934a48e0a897836c544a4b548d7b8e4a51673ef3deb82411c0008d0697092b2"}
Jul 13 20:10:28.105906 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:28.105723    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a934a48e0a897836c544a4b548d7b8e4a51673ef3deb82411c0008d0697092b2"
Jul 13 20:10:50.691146 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:50.691102    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-83eaac-qgmpj"]
Jul 13 20:10:50.691675 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:50.691457    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47be563a-cf47-4eb0-b04c-72e673ba4996" containerName="revoke-keys"
Jul 13 20:10:50.691675 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:50.691467    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="47be563a-cf47-4eb0-b04c-72e673ba4996" containerName="revoke-keys"
Jul 13 20:10:50.691675 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:50.691544    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="47be563a-cf47-4eb0-b04c-72e673ba4996" containerName="revoke-keys"
Jul 13 20:10:50.695722 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:50.695701    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-83eaac-qgmpj"
Jul 13 20:10:50.700625 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:50.700597    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 20:10:50.700792 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:50.700604    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q5n9g\""
Jul 13 20:10:50.704377 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:50.704352    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-83eaac-qgmpj"]
Jul 13 20:10:50.811992 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:50.811946    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc2c18a1-1314-4da3-9c87-94b3ba679bd5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-83eaac-qgmpj\" (UID: \"dc2c18a1-1314-4da3-9c87-94b3ba679bd5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-83eaac-qgmpj"
Jul 13 20:10:50.913110 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:50.913067    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc2c18a1-1314-4da3-9c87-94b3ba679bd5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-83eaac-qgmpj\" (UID: \"dc2c18a1-1314-4da3-9c87-94b3ba679bd5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-83eaac-qgmpj"
Jul 13 20:10:50.913749 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:50.913729    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc2c18a1-1314-4da3-9c87-94b3ba679bd5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-83eaac-qgmpj\" (UID: \"dc2c18a1-1314-4da3-9c87-94b3ba679bd5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-83eaac-qgmpj"
Jul 13 20:10:51.007445 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:51.007350    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-83eaac-qgmpj"
Jul 13 20:10:51.151909 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:51.151878    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-83eaac-qgmpj"]
Jul 13 20:10:51.154018 ip-10-0-143-91 kubenswrapper[2573]: W0713 20:10:51.153989    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc2c18a1_1314_4da3_9c87_94b3ba679bd5.slice/crio-f38c9979a0f88bc3e5b781d42591d85c975fcf3f37374159a29a8250952c5b04 WatchSource:0}: Error finding container f38c9979a0f88bc3e5b781d42591d85c975fcf3f37374159a29a8250952c5b04: Status 404 returned error can't find the container with id f38c9979a0f88bc3e5b781d42591d85c975fcf3f37374159a29a8250952c5b04
Jul 13 20:10:51.223873 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:51.223843    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-83eaac-qgmpj" event={"ID":"dc2c18a1-1314-4da3-9c87-94b3ba679bd5","Type":"ContainerStarted","Data":"f38c9979a0f88bc3e5b781d42591d85c975fcf3f37374159a29a8250952c5b04"}
Jul 13 20:10:52.231050 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:52.231010    2573 generic.go:356] "Generic (PLEG): container finished" podID="dc2c18a1-1314-4da3-9c87-94b3ba679bd5" containerID="1e4975fadc2dd90b4731c66aedcb1cf11e15dbdf1713c55e539de50639625934" exitCode=0
Jul 13 20:10:52.231461 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:52.231073    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-83eaac-qgmpj" event={"ID":"dc2c18a1-1314-4da3-9c87-94b3ba679bd5","Type":"ContainerDied","Data":"1e4975fadc2dd90b4731c66aedcb1cf11e15dbdf1713c55e539de50639625934"}
Jul 13 20:10:53.372954 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:53.372922    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-83eaac-qgmpj"
Jul 13 20:10:53.538744 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:53.538650    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc2c18a1-1314-4da3-9c87-94b3ba679bd5-maas-api-service-ca\") pod \"dc2c18a1-1314-4da3-9c87-94b3ba679bd5\" (UID: \"dc2c18a1-1314-4da3-9c87-94b3ba679bd5\") "
Jul 13 20:10:53.539038 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:53.539013    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc2c18a1-1314-4da3-9c87-94b3ba679bd5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dc2c18a1-1314-4da3-9c87-94b3ba679bd5" (UID: "dc2c18a1-1314-4da3-9c87-94b3ba679bd5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 20:10:53.639720 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:53.639683    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc2c18a1-1314-4da3-9c87-94b3ba679bd5-maas-api-service-ca\") on node \"ip-10-0-143-91.ec2.internal\" DevicePath \"\""
Jul 13 20:10:54.242807 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:54.242770    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-83eaac-qgmpj" event={"ID":"dc2c18a1-1314-4da3-9c87-94b3ba679bd5","Type":"ContainerDied","Data":"f38c9979a0f88bc3e5b781d42591d85c975fcf3f37374159a29a8250952c5b04"}
Jul 13 20:10:54.242807 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:54.242808    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-83eaac-qgmpj"
Jul 13 20:10:54.243035 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:10:54.242827    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f38c9979a0f88bc3e5b781d42591d85c975fcf3f37374159a29a8250952c5b04"
Jul 13 20:11:17.155950 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:17.155903    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6dc479448c-rsbcn_d751e3ec-82ca-4020-8c9a-922ba248b1ca/authorino/0.log"
Jul 13 20:11:22.612624 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:22.612585    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-k9wqz_d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75/manager/0.log"
Jul 13 20:11:22.741875 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:22.741838    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6996d85cfb-6mstv_350b25d0-f70f-4ff6-97df-1b1325c4bfed/manager/0.log"
Jul 13 20:11:22.764583 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:22.764557    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6996d85cfb-6mstv_350b25d0-f70f-4ff6-97df-1b1325c4bfed/copy-manifests/0.log"
Jul 13 20:11:23.463494 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:23.463464    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29732865-rx4qh_c5da24b7-3b69-483d-90af-01daac639565/cleanup/0.log"
Jul 13 20:11:23.569906 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:23.569877    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29732880-4fdvj_8be5d623-00b3-46ba-bc0f-c2cc35065ba9/cleanup/0.log"
Jul 13 20:11:23.681740 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:23.681709    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-1b0d941e-nctjc_288e10bd-518c-482d-92dd-5d655f887880/revoke-keys/0.log"
Jul 13 20:11:23.794973 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:23.794885    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-32278165-q6lfr_54b09556-56e8-429f-b17f-b69d24c45583/revoke-keys/0.log"
Jul 13 20:11:23.905157 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:23.905125    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-519ff3-d9ddk_1d3ba106-dc46-4d92-aa60-52803d063c40/revoke-keys/0.log"
Jul 13 20:11:24.016619 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:24.016582    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-f221d9-mpzlw_ec52eb22-00ab-435f-a826-87327f559716/revoke-keys/0.log"
Jul 13 20:11:24.122880 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:24.122800    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-7a28704c-wrrpl_a0853e99-6cb4-4861-9888-511d92bc2c17/revoke-keys/0.log"
Jul 13 20:11:24.249850 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:24.249819    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-dffe11-p8zlh_ea1d6bff-86fb-48fb-a821-61ac3fbde62f/revoke-keys/0.log"
Jul 13 20:11:24.373030 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:24.372945    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-7b8c26-zs7pl_27538973-2037-4ce5-bd50-4feb6c35f108/revoke-keys/0.log"
Jul 13 20:11:24.483707 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:24.483670    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-f3b1e94a-2r42d_a5f30572-8152-49a7-921c-d0c889b06dcf/revoke-keys/0.log"
Jul 13 20:11:24.594854 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:24.594828    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-ccb43b5e-5swpj_da309781-4f9f-405e-b2d1-63cfa6b17284/revoke-keys/0.log"
Jul 13 20:11:24.812932 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:24.812903    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-452240-v4x8z_47be563a-cf47-4eb0-b04c-72e673ba4996/revoke-keys/0.log"
Jul 13 20:11:24.920093 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:24.920067    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-83eaac-qgmpj_dc2c18a1-1314-4da3-9c87-94b3ba679bd5/revoke-keys/0.log"
Jul 13 20:11:26.377102 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:26.377070    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6dc479448c-rsbcn_d751e3ec-82ca-4020-8c9a-922ba248b1ca/authorino/0.log"
Jul 13 20:11:26.502542 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:26.502474    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-pxkvw_e341a781-5754-4b8b-ab7d-a09308d95354/manager/0.log"
Jul 13 20:11:26.723942 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:26.723912    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-prjbj_40dbad3e-3816-4a6e-96ac-368f118495fe/kuadrant-console-plugin/0.log"
Jul 13 20:11:27.786537 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:27.786501    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55f6586c87-gbf8l_6a67ca80-5190-41d6-96d0-5bdc1047fc6f/kube-auth-proxy/0.log"
Jul 13 20:11:28.398604 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:28.398566    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68dc6f48c8-qx2hp_a55203b2-d840-4319-9ced-26799e123698/router/0.log"
Jul 13 20:11:29.093474 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:29.093442    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db67jz9_6f92caa9-f1be-4633-b300-ca3f1f874d45/main/0.log"
Jul 13 20:11:29.325467 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:29.325429    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-k4cs7_a8d34aef-a8fd-4b33-a587-f619f1e62d59/main/0.log"
Jul 13 20:11:32.986830 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:32.986792    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dl6jf/must-gather-nlxw9"]
Jul 13 20:11:32.987742 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:32.987717    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc2c18a1-1314-4da3-9c87-94b3ba679bd5" containerName="revoke-keys"
Jul 13 20:11:32.987742 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:32.987745    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc2c18a1-1314-4da3-9c87-94b3ba679bd5" containerName="revoke-keys"
Jul 13 20:11:32.987926 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:32.987916    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc2c18a1-1314-4da3-9c87-94b3ba679bd5" containerName="revoke-keys"
Jul 13 20:11:32.992479 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:32.992446    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dl6jf/must-gather-nlxw9"
Jul 13 20:11:32.995691 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:32.995668    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dl6jf\"/\"default-dockercfg-pfg74\""
Jul 13 20:11:32.995793 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:32.995698    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dl6jf\"/\"openshift-service-ca.crt\""
Jul 13 20:11:32.996734 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:32.996683    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dl6jf\"/\"kube-root-ca.crt\""
Jul 13 20:11:32.999523 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:32.999498    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dl6jf/must-gather-nlxw9"]
Jul 13 20:11:33.076693 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:33.076623    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/33f190a0-0be2-4d57-8334-2c209ce160bb-must-gather-output\") pod \"must-gather-nlxw9\" (UID: \"33f190a0-0be2-4d57-8334-2c209ce160bb\") " pod="openshift-must-gather-dl6jf/must-gather-nlxw9"
Jul 13 20:11:33.076885 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:33.076729    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89p75\" (UniqueName: \"kubernetes.io/projected/33f190a0-0be2-4d57-8334-2c209ce160bb-kube-api-access-89p75\") pod \"must-gather-nlxw9\" (UID: \"33f190a0-0be2-4d57-8334-2c209ce160bb\") " pod="openshift-must-gather-dl6jf/must-gather-nlxw9"
Jul 13 20:11:33.177941 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:33.177900    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/33f190a0-0be2-4d57-8334-2c209ce160bb-must-gather-output\") pod \"must-gather-nlxw9\" (UID: \"33f190a0-0be2-4d57-8334-2c209ce160bb\") " pod="openshift-must-gather-dl6jf/must-gather-nlxw9"
Jul 13 20:11:33.178146 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:33.177968    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89p75\" (UniqueName: \"kubernetes.io/projected/33f190a0-0be2-4d57-8334-2c209ce160bb-kube-api-access-89p75\") pod \"must-gather-nlxw9\" (UID: \"33f190a0-0be2-4d57-8334-2c209ce160bb\") " pod="openshift-must-gather-dl6jf/must-gather-nlxw9"
Jul 13 20:11:33.178379 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:33.178355    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/33f190a0-0be2-4d57-8334-2c209ce160bb-must-gather-output\") pod \"must-gather-nlxw9\" (UID: \"33f190a0-0be2-4d57-8334-2c209ce160bb\") " pod="openshift-must-gather-dl6jf/must-gather-nlxw9"
Jul 13 20:11:33.196913 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:33.196879    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89p75\" (UniqueName: \"kubernetes.io/projected/33f190a0-0be2-4d57-8334-2c209ce160bb-kube-api-access-89p75\") pod \"must-gather-nlxw9\" (UID: \"33f190a0-0be2-4d57-8334-2c209ce160bb\") " pod="openshift-must-gather-dl6jf/must-gather-nlxw9"
Jul 13 20:11:33.303792 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:33.303708    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dl6jf/must-gather-nlxw9"
Jul 13 20:11:33.439702 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:33.439671    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dl6jf/must-gather-nlxw9"]
Jul 13 20:11:33.441258 ip-10-0-143-91 kubenswrapper[2573]: W0713 20:11:33.441231    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33f190a0_0be2_4d57_8334_2c209ce160bb.slice/crio-73e7664cbfb9681f1ff7adb4cef3a59892be05fa5f221eaf5949075d6ee1e659 WatchSource:0}: Error finding container 73e7664cbfb9681f1ff7adb4cef3a59892be05fa5f221eaf5949075d6ee1e659: Status 404 returned error can't find the container with id 73e7664cbfb9681f1ff7adb4cef3a59892be05fa5f221eaf5949075d6ee1e659
Jul 13 20:11:34.449683 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:34.449630    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dl6jf/must-gather-nlxw9" event={"ID":"33f190a0-0be2-4d57-8334-2c209ce160bb","Type":"ContainerStarted","Data":"73e7664cbfb9681f1ff7adb4cef3a59892be05fa5f221eaf5949075d6ee1e659"}
Jul 13 20:11:35.458134 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:35.458086    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dl6jf/must-gather-nlxw9" event={"ID":"33f190a0-0be2-4d57-8334-2c209ce160bb","Type":"ContainerStarted","Data":"5d00315a673ce43aefa61b77da1786f60450ebf0ea06cdf64072379461a08b8d"}
Jul 13 20:11:35.458504 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:35.458145    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dl6jf/must-gather-nlxw9" event={"ID":"33f190a0-0be2-4d57-8334-2c209ce160bb","Type":"ContainerStarted","Data":"261d417d21c820e3abc77939726e1885240665cd5f8e88915b49ccb5ddbfe595"}
Jul 13 20:11:35.476060 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:35.475994    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dl6jf/must-gather-nlxw9" podStartSLOduration=2.412057837 podStartE2EDuration="3.475972646s" totalImagesPullingTime="1.063914809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:11:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:11:35.474265402 +0000 UTC m=+2992.988142031" watchObservedRunningTime="2026-07-13 20:11:35.475972646 +0000 UTC m=+2992.989849251"
Jul 13 20:11:36.273600 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:36.273562    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xxrt7_690fc787-2061-47e9-8a05-7e596f4ddf0d/global-pull-secret-syncer/0.log"
Jul 13 20:11:36.418927 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:36.418896    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xdp7v_15ae0b99-58a3-4072-beb3-abe52af62c10/konnectivity-agent/0.log"
Jul 13 20:11:36.509707 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:36.509669    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-91.ec2.internal_6db41158752585940a2ece7d4dc21de6/haproxy/0.log"
Jul 13 20:11:42.735645 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:42.735595    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6dc479448c-rsbcn_d751e3ec-82ca-4020-8c9a-922ba248b1ca/authorino/0.log"
Jul 13 20:11:42.810003 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:42.809964    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-pxkvw_e341a781-5754-4b8b-ab7d-a09308d95354/manager/0.log"
Jul 13 20:11:42.890375 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:42.890337    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-prjbj_40dbad3e-3816-4a6e-96ac-368f118495fe/kuadrant-console-plugin/0.log"
Jul 13 20:11:43.171115 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:43.170989    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 20:11:43.172883 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:43.172860    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 20:11:43.176687 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:43.176661    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 20:11:43.186936 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:43.186914    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cj8b5_3d0d821f-0a58-4566-af42-eca2019fa233/ovn-acl-logging/0.log"
Jul 13 20:11:45.254319 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:45.254284    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-gr7gb_c0acc219-19c5-4387-a6d3-7c83cffb00df/cluster-monitoring-operator/0.log"
Jul 13 20:11:45.675774 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:45.675660    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xfkgj_29014e6c-7b56-4cba-8291-567036c8047b/node-exporter/0.log"
Jul 13 20:11:45.705429 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:45.705401    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xfkgj_29014e6c-7b56-4cba-8291-567036c8047b/kube-rbac-proxy/0.log"
Jul 13 20:11:45.736547 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:45.736520    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xfkgj_29014e6c-7b56-4cba-8291-567036c8047b/init-textfile/0.log"
Jul 13 20:11:45.890450 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:45.890417    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_17fa1a59-da82-416c-904c-61c8c9a8b4f6/prometheus/0.log"
Jul 13 20:11:45.917694 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:45.917664    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_17fa1a59-da82-416c-904c-61c8c9a8b4f6/config-reloader/0.log"
Jul 13 20:11:45.944287 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:45.944255    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_17fa1a59-da82-416c-904c-61c8c9a8b4f6/thanos-sidecar/0.log"
Jul 13 20:11:45.977335 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:45.977278    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_17fa1a59-da82-416c-904c-61c8c9a8b4f6/kube-rbac-proxy-web/0.log"
Jul 13 20:11:46.003973 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:46.003943    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_17fa1a59-da82-416c-904c-61c8c9a8b4f6/kube-rbac-proxy/0.log"
Jul 13 20:11:46.037148 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:46.037119    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_17fa1a59-da82-416c-904c-61c8c9a8b4f6/kube-rbac-proxy-thanos/0.log"
Jul 13 20:11:46.062003 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:46.061965    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_17fa1a59-da82-416c-904c-61c8c9a8b4f6/init-config-reloader/0.log"
Jul 13 20:11:47.470053 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:47.470012    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-dqtq2_54bb1498-3f40-47ca-a33c-eaebcf72090b/networking-console-plugin/0.log"
Jul 13 20:11:47.685282 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:47.685241    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9"]
Jul 13 20:11:47.691035 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:47.691008    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9"
Jul 13 20:11:47.702616 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:47.702566    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9"]
Jul 13 20:11:47.831493 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:47.831416    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fd583187-5986-4d2a-8ac1-23e8cbb2ff39-proc\") pod \"perf-node-gather-daemonset-t8kb9\" (UID: \"fd583187-5986-4d2a-8ac1-23e8cbb2ff39\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9"
Jul 13 20:11:47.831493 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:47.831462    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fd583187-5986-4d2a-8ac1-23e8cbb2ff39-podres\") pod \"perf-node-gather-daemonset-t8kb9\" (UID: \"fd583187-5986-4d2a-8ac1-23e8cbb2ff39\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9"
Jul 13 20:11:47.831750 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:47.831621    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs626\" (UniqueName: \"kubernetes.io/projected/fd583187-5986-4d2a-8ac1-23e8cbb2ff39-kube-api-access-zs626\") pod \"perf-node-gather-daemonset-t8kb9\" (UID: \"fd583187-5986-4d2a-8ac1-23e8cbb2ff39\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9"
Jul 13 20:11:47.831750 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:47.831709    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fd583187-5986-4d2a-8ac1-23e8cbb2ff39-lib-modules\") pod \"perf-node-gather-daemonset-t8kb9\" (UID: \"fd583187-5986-4d2a-8ac1-23e8cbb2ff39\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9"
Jul 13 20:11:47.831854 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:47.831769    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd583187-5986-4d2a-8ac1-23e8cbb2ff39-sys\") pod \"perf-node-gather-daemonset-t8kb9\" (UID: \"fd583187-5986-4d2a-8ac1-23e8cbb2ff39\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9"
Jul 13 20:11:47.932293 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:47.932260    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fd583187-5986-4d2a-8ac1-23e8cbb2ff39-proc\") pod \"perf-node-gather-daemonset-t8kb9\" (UID: \"fd583187-5986-4d2a-8ac1-23e8cbb2ff39\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9"
Jul 13 20:11:47.932483 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:47.932301    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fd583187-5986-4d2a-8ac1-23e8cbb2ff39-podres\") pod \"perf-node-gather-daemonset-t8kb9\" (UID: \"fd583187-5986-4d2a-8ac1-23e8cbb2ff39\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9"
Jul 13 20:11:47.932483 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:47.932345    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs626\" (UniqueName: \"kubernetes.io/projected/fd583187-5986-4d2a-8ac1-23e8cbb2ff39-kube-api-access-zs626\") pod \"perf-node-gather-daemonset-t8kb9\" (UID: \"fd583187-5986-4d2a-8ac1-23e8cbb2ff39\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9"
Jul 13 20:11:47.932483 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:47.932378    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fd583187-5986-4d2a-8ac1-23e8cbb2ff39-lib-modules\") pod \"perf-node-gather-daemonset-t8kb9\" (UID: \"fd583187-5986-4d2a-8ac1-23e8cbb2ff39\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9"
Jul 13 20:11:47.932483 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:47.932387    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fd583187-5986-4d2a-8ac1-23e8cbb2ff39-proc\") pod \"perf-node-gather-daemonset-t8kb9\" (UID: \"fd583187-5986-4d2a-8ac1-23e8cbb2ff39\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9"
Jul 13 20:11:47.932483 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:47.932418    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd583187-5986-4d2a-8ac1-23e8cbb2ff39-sys\") pod \"perf-node-gather-daemonset-t8kb9\" (UID: \"fd583187-5986-4d2a-8ac1-23e8cbb2ff39\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9"
Jul 13 20:11:47.932749 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:47.932503    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd583187-5986-4d2a-8ac1-23e8cbb2ff39-sys\") pod \"perf-node-gather-daemonset-t8kb9\" (UID: \"fd583187-5986-4d2a-8ac1-23e8cbb2ff39\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9"
Jul 13 20:11:47.932749 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:47.932511    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fd583187-5986-4d2a-8ac1-23e8cbb2ff39-podres\") pod \"perf-node-gather-daemonset-t8kb9\" (UID: \"fd583187-5986-4d2a-8ac1-23e8cbb2ff39\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9"
Jul 13 20:11:47.932749 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:47.932572    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fd583187-5986-4d2a-8ac1-23e8cbb2ff39-lib-modules\") pod \"perf-node-gather-daemonset-t8kb9\" (UID: \"fd583187-5986-4d2a-8ac1-23e8cbb2ff39\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9"
Jul 13 20:11:47.941891 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:47.941861    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs626\" (UniqueName: \"kubernetes.io/projected/fd583187-5986-4d2a-8ac1-23e8cbb2ff39-kube-api-access-zs626\") pod \"perf-node-gather-daemonset-t8kb9\" (UID: \"fd583187-5986-4d2a-8ac1-23e8cbb2ff39\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9"
Jul 13 20:11:48.004908 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:48.004868    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9"
Jul 13 20:11:48.091653 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:48.089375    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/2.log"
Jul 13 20:11:48.100982 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:48.100872    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-c55st_319e53f6-7d9e-48f3-b9f6-001ada635739/console-operator/3.log"
Jul 13 20:11:48.198074 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:48.198038    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9"]
Jul 13 20:11:48.207788 ip-10-0-143-91 kubenswrapper[2573]: W0713 20:11:48.207735    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfd583187_5986_4d2a_8ac1_23e8cbb2ff39.slice/crio-b8ec498ded8e49e5ccee709307d7454b20cf7734f1226f6617bd2e38ed570642 WatchSource:0}: Error finding container b8ec498ded8e49e5ccee709307d7454b20cf7734f1226f6617bd2e38ed570642: Status 404 returned error can't find the container with id b8ec498ded8e49e5ccee709307d7454b20cf7734f1226f6617bd2e38ed570642
Jul 13 20:11:48.562718 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:48.562679    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9" event={"ID":"fd583187-5986-4d2a-8ac1-23e8cbb2ff39","Type":"ContainerStarted","Data":"f54b4360c43d6e0a6c6124de0ccf3cc380d3efe8b4a82d84bfd952b67faa3dde"}
Jul 13 20:11:48.563249 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:48.562751    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9" event={"ID":"fd583187-5986-4d2a-8ac1-23e8cbb2ff39","Type":"ContainerStarted","Data":"b8ec498ded8e49e5ccee709307d7454b20cf7734f1226f6617bd2e38ed570642"}
Jul 13 20:11:48.563249 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:48.562780    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9"
Jul 13 20:11:48.579304 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:48.579243    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9" podStartSLOduration=1.579219342 podStartE2EDuration="1.579219342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:11:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:11:48.578450943 +0000 UTC m=+3006.092327578" watchObservedRunningTime="2026-07-13 20:11:48.579219342 +0000 UTC m=+3006.093095946"
Jul 13 20:11:48.684354 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:48.684311    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-jxg7k_1c3603c3-d25d-4ab4-9708-213b4c3aeb88/download-server/0.log"
Jul 13 20:11:49.298102 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:49.298061    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-mmdms_507498fd-317d-48a0-b218-ea88bf7e6a58/volume-data-source-validator/0.log"
Jul 13 20:11:50.569235 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:50.569200    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-s9z7s_a0fce8f4-676e-4fec-adf5-09b3fd26434d/dns/0.log"
Jul 13 20:11:50.594553 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:50.594522    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-s9z7s_a0fce8f4-676e-4fec-adf5-09b3fd26434d/kube-rbac-proxy/0.log"
Jul 13 20:11:50.697837 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:50.697769    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6kk2c_9d300bbe-5196-46b2-96a3-a35f690f0207/dns-node-resolver/0.log"
Jul 13 20:11:51.375990 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:51.375961    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-l6bw8_11a0b6c6-aae7-459a-9463-95459fc8ab61/node-ca/0.log"
Jul 13 20:11:52.757835 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:52.757797    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55f6586c87-gbf8l_6a67ca80-5190-41d6-96d0-5bdc1047fc6f/kube-auth-proxy/0.log"
Jul 13 20:11:52.973662 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:52.973610    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68dc6f48c8-qx2hp_a55203b2-d840-4319-9ced-26799e123698/router/0.log"
Jul 13 20:11:53.747780 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:53.747753    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-59rng_95371626-7fce-48f7-a5ba-91cc14074e85/serve-healthcheck-canary/0.log"
Jul 13 20:11:54.345898 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:54.345856    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-nnsb7_c5945d50-bd96-4904-96ba-ea69aaef93d8/insights-operator/0.log"
Jul 13 20:11:54.347295 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:54.347269    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-nnsb7_c5945d50-bd96-4904-96ba-ea69aaef93d8/insights-operator/1.log"
Jul 13 20:11:54.460104 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:54.460074    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ktrc2_492fd2c6-a58d-4a24-9a2f-1911b5747a0d/kube-rbac-proxy/0.log"
Jul 13 20:11:54.483000 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:54.482968    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ktrc2_492fd2c6-a58d-4a24-9a2f-1911b5747a0d/exporter/0.log"
Jul 13 20:11:54.506260 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:54.506230    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ktrc2_492fd2c6-a58d-4a24-9a2f-1911b5747a0d/extractor/0.log"
Jul 13 20:11:54.580937 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:54.580910    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-t8kb9"
Jul 13 20:11:57.460693 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:57.460654    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-k9wqz_d092db6a-ddb6-46e4-ac4c-4c9f8e39ef75/manager/0.log"
Jul 13 20:11:57.585105 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:57.585073    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6996d85cfb-6mstv_350b25d0-f70f-4ff6-97df-1b1325c4bfed/manager/0.log"
Jul 13 20:11:57.587282 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:57.587253    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55f6586c87-gbf8l_6a67ca80-5190-41d6-96d0-5bdc1047fc6f/kube-auth-proxy/0.log"
Jul 13 20:11:57.608757 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:57.608727    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6996d85cfb-6mstv_350b25d0-f70f-4ff6-97df-1b1325c4bfed/copy-manifests/0.log"
Jul 13 20:11:57.747231 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:57.747185    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68dc6f48c8-qx2hp_a55203b2-d840-4319-9ced-26799e123698/router/0.log"
Jul 13 20:11:59.003049 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:59.003014    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-655465cc8d-kqlmd_d98a29a9-caf1-4db4-a7d3-bfb08bc4a40d/manager/0.log"
Jul 13 20:11:59.229104 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:59.229071    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55f6586c87-gbf8l_6a67ca80-5190-41d6-96d0-5bdc1047fc6f/kube-auth-proxy/0.log"
Jul 13 20:11:59.387993 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:11:59.387954    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68dc6f48c8-qx2hp_a55203b2-d840-4319-9ced-26799e123698/router/0.log"
Jul 13 20:12:04.835411 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:12:04.835316    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-p47mr_f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc/kube-storage-version-migrator-operator/1.log"
Jul 13 20:12:04.836861 ip-10-0-143-91 kubenswrapper[2573]: I0713 20:12:04.836833    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-p47mr_f6c9fcfe-e75f-4c3d-bbf0-baee21960fbc/kube-storage-version-migrator-operator/0.log"